-- Logs begin at Wed 2022-07-13 08:16:21 UTC, end at Wed 2022-07-13 09:25:08 UTC. -- Jul 13 08:46:36.471374 dv-cinder-849626-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 13 08:46:36.480303 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 13 08:46:36 2022] *** Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: nodename: dv-cinder-849626-3-iscsi Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: machine: x86_64 Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: clock source: unix Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: pcre jit disabled Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: detected number of CPU cores: 4 Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: current working directory: / Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: detected binary path: /usr/bin/uwsgi-core Jul 13 08:46:36.482353 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: your processes number limit is 47680 Jul 13 08:46:36.482949 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: your memory page size is 4096 bytes Jul 13 08:46:36.482949 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: detected max file descriptor number: 2048 Jul 13 08:46:36.482949 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: lock engine: pthread robust mutexes Jul 13 08:46:36.482949 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: thunder lock: enabled Jul 13 08:46:36.482949 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 13 08:46:36.482949 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 13 08:46:36.527754 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: Python main interpreter initialized at 0x561035e6d9c0 Jul 13 08:46:36.527754 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: python threads support enabled Jul 13 08:46:36.527754 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: your server socket listen backlog is limited to 100 connections Jul 13 08:46:36.528055 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: your mercy for graceful operations on workers is 90 seconds Jul 13 08:46:36.528140 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: mapped 403077 bytes (393 KB) for 2 cores Jul 13 08:46:36.528217 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: *** Operational MODE: preforking *** Jul 13 08:46:36.528476 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: *** uWSGI is running in multiple interpreter mode *** Jul 13 08:46:36.528476 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: spawned uWSGI master process (pid: 80276) Jul 13 08:46:36.528673 dv-cinder-849626-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 13 08:46:36.530924 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: spawned uWSGI worker 1 (pid: 80287, cores: 1) Jul 13 08:46:36.530924 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: spawned uWSGI worker 2 (pid: 80288, cores: 1) Jul 13 08:46:36.530924 dv-cinder-849626-3-iscsi devstack@keystone.service[80276]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 13 08:46:41.957304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 08:46:41.957304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: from cryptography.utils import int_from_bytes Jul 13 08:46:41.958612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 08:46:41.958612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: from cryptography.utils import int_from_bytes Jul 13 08:46:41.958727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 08:46:41.958727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: from cryptography.utils import int_from_bytes Jul 13 08:46:41.958727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 08:46:41.958727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: from cryptography.utils import int_from_bytes Jul 13 08:46:42.007260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 13 08:46:42.007461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 13 08:46:42.007694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] command line args: [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 13 08:46:42.007845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 13 08:46:42.007976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 13 08:46:42.008060 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 13 08:46:42.008143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ================================================================================ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 13 08:46:42.008224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] command line args: [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 13 08:46:42.008395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] admin_token = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.008473 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 13 08:46:42.008641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config_dir = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.008724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ================================================================================ {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 13 08:46:42.008828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.008906 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] admin_token = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.009011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config_source = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.009130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] config_dir = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.009208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] debug = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.009320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.009407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.009667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] config_source = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.009667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] debug = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.009867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.009969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.010134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.010234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.010333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.010434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.010548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.010653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.010755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.010854 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.010954 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_config_append = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.010954 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.011145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.011249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_dir = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.011350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_config_append = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.011449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.011543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.011543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_dir = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.011707 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.011830 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_file = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.011911 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012784 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.012946 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.013060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.013135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.013252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.013355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.013442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.013590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.013662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.013825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.013908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.014140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.014278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.014396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.014547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.014666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.014754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.014932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.015159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.015239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.015342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] publish_errors = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.015574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.015742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.015859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.015934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] publish_errors = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.016044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.016126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.016198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.016316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.016450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.016588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.016674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] standard_threads = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.016837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.016914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017178 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] standard_threads = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_journal = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_json = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_stderr = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.017928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_syslog = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.018003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] use_journal = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.018177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] use_json = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.018260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.018410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] use_stderr = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.018484 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.018554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] use_syslog = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.018697 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.018780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 08:46:42.018954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.019044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.019195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.019271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.019444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.019521 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.019682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.019770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.019929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.020012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.020167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.020243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.020449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.020544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.020696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.020847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.020971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.021100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.021183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.021350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.021434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.021598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.021680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.021835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.021911 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.022106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.022201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.022341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.022470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.022596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.022729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.022850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.022982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.023096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.023222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.023332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.023479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.023584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.023735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.023948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.024024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.024207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.024357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.024442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.024513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.024681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.024816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.024912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.025078 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.025155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.025332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.025414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.025569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.025644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.025808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.025886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.026078 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.026159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.026300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.026376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.026548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.026626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.026797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.026878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.027031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.027107 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.027264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.027341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.027502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.027603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.027739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.027821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.027976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.028063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.028206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.028284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.028437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.028532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.028667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.028752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.028899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.028978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.029129 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.029206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.029362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.029441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.029593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.029672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.029824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.029907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.030072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.030155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.037939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038413 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038863 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.038989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.039935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040005 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.040957 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.041957 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.042851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043620 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.043921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044533 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.external = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044838 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.044951 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.external = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.password = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.token = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045424 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.password = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045619 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.token = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.045999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.046085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.046147 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.046205 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.046344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.047153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.047228 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.047291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.047356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.047421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.047481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.047539 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.047605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.047666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.047784 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.048325 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.048393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.048464 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.048537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.048599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.048658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.048812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.048970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.049050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.049132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.049212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.049286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.049357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.049428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.049575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.049676 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.049756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.049958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.050067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.050230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.050319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.050490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.050568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.050732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.050871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.050978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.051072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.051210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.051288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.051446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.052012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.052237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.052326 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.052496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.052576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.052726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.052806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.052975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.053064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.053209 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.053288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.053481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.053664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.053807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.053918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.054000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.054096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.054249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.054330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.054498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.054589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.054738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.054852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.054989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.055124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.055237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.055359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.055476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.055598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.055714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.055849 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.055963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.056104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.056183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.056342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.056421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.056576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.056654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.056807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.056886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.057066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.057145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.057316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.057394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.057558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.057636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.057807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.057891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.058077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.058189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.058332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.058421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.058582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.058659 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.058820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.058899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.059052 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.059132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.059289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.059367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.059519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.059606 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.059755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.059833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.059996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.060074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.060229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.060339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.060466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.060548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.060694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.060772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.060937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.061028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.061182 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.061261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.061411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.061488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.061684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.061780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.061903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.061985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.062184 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.062273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.062396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066471 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.066948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.067015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.067077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.067226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.067820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.067891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.067952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.068985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069110 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069406 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069533 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069896 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.069955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.070013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.070390 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.070454 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.070527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.070632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.070713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.070786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.070857 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.070929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071009 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.071964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.072982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.073133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.073212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.073379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.073536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.073612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.073684 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.073842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.073921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.074110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.074205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.074360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.074439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.074604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.074683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.074837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.074927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.075073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.075153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.075315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.075394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.075568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.075648 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.075805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.075884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.076040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.076118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.076288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.076367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.076537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.076614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.076774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.076851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.077017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.077102 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.077255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.077334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.077491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.077568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.077728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.077808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.077974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.078074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.078233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.078312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.078469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] role.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.078549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.078719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] role.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.078797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.078952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.079030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.079203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.079283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.079450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.079527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.079782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.079889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.079978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.080055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.080179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.080259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.080409 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.080488 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.080657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.080737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.080904 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.080983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.081152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.081230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.081407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.081487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.081655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.081821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.081899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.082074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.082170 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.082328 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.082560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.082638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.082933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.083216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.083299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.083371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.083371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.083500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.083571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.083677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.083784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.083930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.084009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.084167 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.084245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.084424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.084508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.084678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.084757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.084919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.084997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.085153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.085233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.085388 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.085465 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.085618 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.085696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.085857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.085935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.086116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.086197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.086358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.086435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.086596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.086677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.086845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.086930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.087076 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.087154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.087317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.087396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.087586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.087879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.087966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.088088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.088175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.088249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.088320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.088391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.088501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.088580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.088739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.088819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.088977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.089055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.089214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.089292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.089461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.089541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.089710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.089788 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.089939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.090020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.090201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.090281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.090450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.090529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.090701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.090779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.090934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.091025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.091203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.091293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.091436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.091514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.091679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.091765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.connection = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.091916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.092007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.092153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.092232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.092388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.092466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.092623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.092808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.092886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.092947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.093091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.093169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.093333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.093419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.093569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.093647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.093815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.093892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.094070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.098695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.098795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.098874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.098948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.099011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.099011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.099138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.099213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.099285 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.099358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 13 08:46:42.099434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.099434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.099434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.099434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 08:46:42.099434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 13 08:46:42.126507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.126755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.126963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.127119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.127356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.127589 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.136208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.136717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.164583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.164809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.164989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.165223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.308623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.308804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.357657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.357893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.358066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.358210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.422918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.423144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.423259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.423370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.423517 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.423601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.423744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.423827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.423964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.424054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.424183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.424399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.449149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.449384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.449513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.449616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.449750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.449832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.449978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.450086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.450232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.450315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.450444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.450526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.450655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.450867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.461428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.461800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.474546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.474776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.475000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.475111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.475222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.475373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.475455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.475603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.475694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.475829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.475909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.476047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.476263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.476478 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 08:46:42.493569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.493777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 08:46:42.494213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.494414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 08:46:42.494495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.494626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 08:46:42.494708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.494923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.495004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.495148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 08:46:42.495237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.495415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.495497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.495675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.495932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.496187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.506342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.506620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.506887 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.507168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.507467 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.507744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.508013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.508297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.515425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.516635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.518736 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.519015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.519295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.519576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.519856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.519962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.520113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.520243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.520897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.521035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.521316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.521406 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.521593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.521673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.521849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.521929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.522136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.522221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.522405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.522669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.522936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.523199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.549755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.550757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.551393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.551718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.552464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.552740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.556956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.557234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.557631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.558018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.558315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.558750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.564915 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.565197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.565997 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.566304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.569751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.570060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.570333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.570614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.570843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.570939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.571131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.571213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.571396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.571477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.571659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.571742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.571937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.572209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.572476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.572743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.590808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.591082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.591890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.592168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.595597 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.596815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.599822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.600111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.601076 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.601371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.605223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.606557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.608854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.610206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.612501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.612797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.613085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.613369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.613655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.613847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.613939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.614273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.614575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.614861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.615162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.615442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.632959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.633239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.633503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.633773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.634710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.634993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.635260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.635531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.641645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.643375 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.643460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.643650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.645133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.646060 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.647384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.647662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.647943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.648232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.648514 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.649273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.649552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.649834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.650149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.662386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.662663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.662942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.663233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.663511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.664320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.664600 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.664882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.665049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.665169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.665343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.665434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.667015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.667309 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.683610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.685563 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.686906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.688892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.690224 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.691962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.692276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.693729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.694006 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.694114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.694316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.695660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.695941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.696230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.701555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.703463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.703619 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.705464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.706698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.706988 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.707374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.707658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.709385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.709672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.710088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.710370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.717525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.717809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.718112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.719658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.719945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.720223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.726639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.726917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.727196 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.727469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.727735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.728110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.728382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.728664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.728940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.729208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.729425 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 08:46:42.729507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.729788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.729986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.730088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.730282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.730374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.730547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.730930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.731237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.731514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.731791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.732061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.732338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.732602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.732865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.733143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 08:46:42.767958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561035e6d9c0 pid: 80287 (default app) Jul 13 08:46:42.771609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561035e6d9c0 pid: 80288 (default app) Jul 13 08:46:42.771917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff4020-9c84-43b5-9d33-d3f3c8e07620 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:42.772211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff4020-9c84-43b5-9d33-d3f3c8e07620 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:42.772490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff4020-9c84-43b5-9d33-d3f3c8e07620 None None] PATH_INFO: `/v3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:42.776703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Wed Jul 13 08:46:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:46:51.350490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d03aae-95e1-43c8-b3f8-b46e3a8b7215 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:51.350973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d03aae-95e1-43c8-b3f8-b46e3a8b7215 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:51.351065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d03aae-95e1-43c8-b3f8-b46e3a8b7215 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:51.355532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:46:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:46:51.362622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:51.362622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:51.362622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:51.431665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: INFO dbcounter [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] Registered counter for database keystone Jul 13 08:46:51.452897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG oslo_db.sqlalchemy.engines [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80287) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 13 08:46:51.459152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writer thread running {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 13 08:46:51.652999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG passlib.handlers.bcrypt [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80287) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 13 08:46:51.653421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG passlib.handlers.bcrypt [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80287) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 13 08:46:51.707389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:46:51.709536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:46:51.829525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:46:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 13 08:46:52.015704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d03aae-95e1-43c8-b3f8-b46e3a8b7215 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:52.085821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: INFO dbcounter [None req-26d03aae-95e1-43c8-b3f8-b46e3a8b7215 None None] Registered counter for database keystone Jul 13 08:46:52.104602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG oslo_db.sqlalchemy.engines [None req-26d03aae-95e1-43c8-b3f8-b46e3a8b7215 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80288) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 13 08:46:52.110633 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writer thread running {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 13 08:46:52.228849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d03aae-95e1-43c8-b3f8-b46e3a8b7215 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:52.233077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53648d4a-f0a8-4618-80c4-c4a08e06f6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:52.233729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53648d4a-f0a8-4618-80c4-c4a08e06f6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:52.234011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53648d4a-f0a8-4618-80c4-c4a08e06f6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:52.234312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53648d4a-f0a8-4618-80c4-c4a08e06f6d6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:52.242364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53648d4a-f0a8-4618-80c4-c4a08e06f6d6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:52.294163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: WARNING keystone.common.rbac_enforcer.enforcer [None req-53648d4a-f0a8-4618-80c4-c4a08e06f6d6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 13 08:46:52.294444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53648d4a-f0a8-4618-80c4-c4a08e06f6d6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:52.304649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Wed Jul 13 08:46:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 08:46:52.310606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:52.325645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13b59e-0d9d-4833-8267-69d450f43794 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:52.330005 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1726072-279f-4ddd-973e-3e9d82aa8e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:52.331138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1726072-279f-4ddd-973e-3e9d82aa8e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:52.331426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1726072-279f-4ddd-973e-3e9d82aa8e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:52.331702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1726072-279f-4ddd-973e-3e9d82aa8e94 admin admin] PATH_INFO: `/v3/projects/c8052949c90742f6ada3f8a34d5636f8` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:52.335881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1726072-279f-4ddd-973e-3e9d82aa8e94 admin admin] RBAC: Authorizing `identity:get_project(project_id=c8052949c90742f6ada3f8a34d5636f8)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:52.386845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b1726072-279f-4ddd-973e-3e9d82aa8e94 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 13 08:46:52.387344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1726072-279f-4ddd-973e-3e9d82aa8e94 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:52.391825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Wed Jul 13 08:46:52 2022] GET /identity/v3/projects/c8052949c90742f6ada3f8a34d5636f8 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:46:54.386374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b037-9d45-405f-b77d-1a9502deadd1 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:54.386847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b037-9d45-405f-b77d-1a9502deadd1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:54.386938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b037-9d45-405f-b77d-1a9502deadd1 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:54.391065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:46:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:46:54.396962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9cd1-969b-4218-8919-01176d10c980 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:54.397360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9cd1-969b-4218-8919-01176d10c980 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:54.397515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9cd1-969b-4218-8919-01176d10c980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:54.416270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5d8c9cd1-969b-4218-8919-01176d10c980 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:46:54.417776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5d8c9cd1-969b-4218-8919-01176d10c980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:46:54.425739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:46:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 13 08:46:54.448192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b037-9d45-405f-b77d-1a9502deadd1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:54.460769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b037-9d45-405f-b77d-1a9502deadd1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:54.464919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36550089-6ccf-4f24-9b75-cb15c6d502d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:54.465550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36550089-6ccf-4f24-9b75-cb15c6d502d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:54.465831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36550089-6ccf-4f24-9b75-cb15c6d502d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:54.466143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36550089-6ccf-4f24-9b75-cb15c6d502d2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:54.473731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36550089-6ccf-4f24-9b75-cb15c6d502d2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:54.475538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36550089-6ccf-4f24-9b75-cb15c6d502d2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:54.484185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Wed Jul 13 08:46:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 08:46:54.489891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9cd1-969b-4218-8919-01176d10c980 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:54.498871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9cd1-969b-4218-8919-01176d10c980 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:54.503210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff1445-4add-4580-8194-996e85686285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:54.504107 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff1445-4add-4580-8194-996e85686285 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:54.505281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff1445-4add-4580-8194-996e85686285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:54.505600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff1445-4add-4580-8194-996e85686285 admin admin] PATH_INFO: `/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:54.507051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff1445-4add-4580-8194-996e85686285 admin admin] RBAC: Authorizing `identity:get_user(user_id=26ad26ac4aea4a7b8fb923b47a154aa9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:54.508894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff1445-4add-4580-8194-996e85686285 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:54.510491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Wed Jul 13 08:46:54 2022] GET /identity/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:46:56.575409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685395bc-d55c-4303-ad38-ff397896af86 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:56.575940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685395bc-d55c-4303-ad38-ff397896af86 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:56.575940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685395bc-d55c-4303-ad38-ff397896af86 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:56.577293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:46:56.582473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c22fe7-4882-47a1-ae58-ce992be9371c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:56.582977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c22fe7-4882-47a1-ae58-ce992be9371c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:56.583189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c22fe7-4882-47a1-ae58-ce992be9371c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:56.600040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-36c22fe7-4882-47a1-ae58-ce992be9371c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:46:56.601117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-36c22fe7-4882-47a1-ae58-ce992be9371c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:46:56.649264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86500fa1-1955-4fce-8148-90d1c90ccef2 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:56.649545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86500fa1-1955-4fce-8148-90d1c90ccef2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:56.649822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86500fa1-1955-4fce-8148-90d1c90ccef2 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:56.651201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:46:56.656255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4317da-47e6-4b91-b0ca-7d2ee9bf9158 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:56.656540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4317da-47e6-4b91-b0ca-7d2ee9bf9158 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:56.656821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4317da-47e6-4b91-b0ca-7d2ee9bf9158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:56.682153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:46:56.697468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG passlib.handlers.bcrypt [None req-9a4317da-47e6-4b91-b0ca-7d2ee9bf9158 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80288) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 13 08:46:56.697890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG passlib.handlers.bcrypt [None req-9a4317da-47e6-4b91-b0ca-7d2ee9bf9158 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80288) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 13 08:46:56.704971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c22fe7-4882-47a1-ae58-ce992be9371c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:56.728458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:46:56.744780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9a4317da-47e6-4b91-b0ca-7d2ee9bf9158 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:46:56.746420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9a4317da-47e6-4b91-b0ca-7d2ee9bf9158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:46:56.807605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c22fe7-4882-47a1-ae58-ce992be9371c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:56.841276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aa789-6dbc-474c-a400-e70b2be79fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:56.842092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aa789-6dbc-474c-a400-e70b2be79fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:56.842380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aa789-6dbc-474c-a400-e70b2be79fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:56.842656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aa789-6dbc-474c-a400-e70b2be79fc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:56.846554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:46:56.854548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382aa789-6dbc-474c-a400-e70b2be79fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-382aa789-6dbc-474c-a400-e70b2be79fc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:46:56.865901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:46:56.866989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:46:56.867979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:46:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:46:56.869147 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4317da-47e6-4b91-b0ca-7d2ee9bf9158 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:56.874580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aa789-6dbc-474c-a400-e70b2be79fc7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:56.955778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:46:56.970766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4317da-47e6-4b91-b0ca-7d2ee9bf9158 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:56.973045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:46:56.974494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aa789-6dbc-474c-a400-e70b2be79fc7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:57.007883 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084b3a6-2ada-48d1-b0e0-e73dfedbc9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:57.008764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084b3a6-2ada-48d1-b0e0-e73dfedbc9db None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:57.008906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350b7bb-308c-4975-a6c2-25a2cda75e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:57.009047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084b3a6-2ada-48d1-b0e0-e73dfedbc9db None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:57.009319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084b3a6-2ada-48d1-b0e0-e73dfedbc9db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:57.009566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350b7bb-308c-4975-a6c2-25a2cda75e07 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:57.009851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350b7bb-308c-4975-a6c2-25a2cda75e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:57.010171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350b7bb-308c-4975-a6c2-25a2cda75e07 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:57.010922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350b7bb-308c-4975-a6c2-25a2cda75e07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:46:57.011229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350b7bb-308c-4975-a6c2-25a2cda75e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:57.012907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350b7bb-308c-4975-a6c2-25a2cda75e07 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:57.025827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:46:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:46:57.031319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350b7bb-308c-4975-a6c2-25a2cda75e07 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:57.078125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084b3a6-2ada-48d1-b0e0-e73dfedbc9db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:57.079632 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084b3a6-2ada-48d1-b0e0-e73dfedbc9db None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:57.129318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350b7bb-308c-4975-a6c2-25a2cda75e07 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:57.163308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d853f6d-1723-4764-9a80-a508a7f08f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:57.164174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d853f6d-1723-4764-9a80-a508a7f08f37 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:57.164455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d853f6d-1723-4764-9a80-a508a7f08f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:57.164729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d853f6d-1723-4764-9a80-a508a7f08f37 None admin] PATH_INFO: `/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:57.165917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d853f6d-1723-4764-9a80-a508a7f08f37 None admin] RBAC: Authorizing `identity:get_user(user_id=26ad26ac4aea4a7b8fb923b47a154aa9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:57.167418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d853f6d-1723-4764-9a80-a508a7f08f37 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:57.169029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 9/16] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:46:57 2022] GET /identity/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:46:57.174696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d853f6d-1723-4764-9a80-a508a7f08f37 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:57.176538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 8/17] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:46:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:46:57.181889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084b3a6-2ada-48d1-b0e0-e73dfedbc9db None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:57.184622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:46:57.189226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:46:57.275803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d853f6d-1723-4764-9a80-a508a7f08f37 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:57.281934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084b3a6-2ada-48d1-b0e0-e73dfedbc9db None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:57.308811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4a0b4-b2bd-4907-b1fc-5c9788d0383a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:57.309604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4a0b4-b2bd-4907-b1fc-5c9788d0383a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:57.309889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4a0b4-b2bd-4907-b1fc-5c9788d0383a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:57.310196 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4a0b4-b2bd-4907-b1fc-5c9788d0383a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:57.311380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4a0b4-b2bd-4907-b1fc-5c9788d0383a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:57.312833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4a0b4-b2bd-4907-b1fc-5c9788d0383a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:57.314466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 10/18] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:46:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:46:57.315339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495d88-2427-4a2b-9ffa-3d6adbacebb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:57.316123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495d88-2427-4a2b-9ffa-3d6adbacebb1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:57.316401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495d88-2427-4a2b-9ffa-3d6adbacebb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:57.316673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495d88-2427-4a2b-9ffa-3d6adbacebb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:57.320780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4a0b4-b2bd-4907-b1fc-5c9788d0383a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:57.336398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd495d88-2427-4a2b-9ffa-3d6adbacebb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:57.338049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd495d88-2427-4a2b-9ffa-3d6adbacebb1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-cd495d88-2427-4a2b-9ffa-3d6adbacebb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:46:57.361279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:46:57.362268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:46:57.363063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:46:57.363063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:46:57.363063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:46:57.363063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:46:57.363063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 9/19] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:46:57.368279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495d88-2427-4a2b-9ffa-3d6adbacebb1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:46:57.403644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:46:57.422585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4a0b4-b2bd-4907-b1fc-5c9788d0383a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:57.425372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:46:57.456516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd61e25-4bee-4d5c-b0c4-e122f62e1070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:57.457280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd61e25-4bee-4d5c-b0c4-e122f62e1070 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:57.458608 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd61e25-4bee-4d5c-b0c4-e122f62e1070 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:57.458914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd61e25-4bee-4d5c-b0c4-e122f62e1070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:57.459747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd61e25-4bee-4d5c-b0c4-e122f62e1070 None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.domain.id=default, user.id=26ad26ac4aea4a7b8fb923b47a154aa9) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:46:57.460056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd61e25-4bee-4d5c-b0c4-e122f62e1070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:57.461493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd61e25-4bee-4d5c-b0c4-e122f62e1070 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:57.468239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 11/20] 10.222.0.40 () {56 vars in 1322 bytes} [Wed Jul 13 08:46:57 2022] GET /identity/v3/role_assignments?user.id=26ad26ac4aea4a7b8fb923b47a154aa9&scope.domain.id=default&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:46:57.468364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495d88-2427-4a2b-9ffa-3d6adbacebb1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:46:57.503732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881ae9d-02d6-4000-a1e5-15dac4ea5360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:46:57.504936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881ae9d-02d6-4000-a1e5-15dac4ea5360 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:46:57.505252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881ae9d-02d6-4000-a1e5-15dac4ea5360 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:46:57.505531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881ae9d-02d6-4000-a1e5-15dac4ea5360 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:46:57.506433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881ae9d-02d6-4000-a1e5-15dac4ea5360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:46:57.506738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881ae9d-02d6-4000-a1e5-15dac4ea5360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:46:57.508157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881ae9d-02d6-4000-a1e5-15dac4ea5360 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:46:57.528559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 10/21] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:00.071445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b763a-6f8a-4f15-b08a-88bfe5ee830c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.072466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b763a-6f8a-4f15-b08a-88bfe5ee830c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.072967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b763a-6f8a-4f15-b08a-88bfe5ee830c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.074612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 12/22] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:00.081096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d6082-06ea-483f-bdaf-7e24a5c234f5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.081926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d6082-06ea-483f-bdaf-7e24a5c234f5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.083847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d6082-06ea-483f-bdaf-7e24a5c234f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.119116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5c9d6082-06ea-483f-bdaf-7e24a5c234f5 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:00.128265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5c9d6082-06ea-483f-bdaf-7e24a5c234f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:00.144888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380177fd-1c31-4606-8206-a7592e600b80 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.146197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380177fd-1c31-4606-8206-a7592e600b80 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.146698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380177fd-1c31-4606-8206-a7592e600b80 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.148454 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 13/23] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:00.159381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5e31-6a2a-45fc-8406-d965c7f0c6d7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.159980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5e31-6a2a-45fc-8406-d965c7f0c6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.160455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5e31-6a2a-45fc-8406-d965c7f0c6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.198531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-521d5e31-6a2a-45fc-8406-d965c7f0c6d7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:00.201592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-521d5e31-6a2a-45fc-8406-d965c7f0c6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:00.295050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 11/24] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:00.321144 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab8780-5a94-4e5e-8805-4ad03564ab13 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.321435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab8780-5a94-4e5e-8805-4ad03564ab13 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.321767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab8780-5a94-4e5e-8805-4ad03564ab13 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.323105 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 12/25] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:47:00.328262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff31a-6377-4e80-a7d4-85e983f98dfb None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.328570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff31a-6377-4e80-a7d4-85e983f98dfb None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.328853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff31a-6377-4e80-a7d4-85e983f98dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.358128 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ad3ff31a-6377-4e80-a7d4-85e983f98dfb None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:00.359316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ad3ff31a-6377-4e80-a7d4-85e983f98dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:00.365163 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 14/26] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:00.366243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d5e31-6a2a-45fc-8406-d965c7f0c6d7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:00.456523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:00.505110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:00.516497 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 13/27] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:00.519231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff31a-6377-4e80-a7d4-85e983f98dfb None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:00.554426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d5e31-6a2a-45fc-8406-d965c7f0c6d7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:00.595464 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a2219-a121-4216-870e-b40883a2ae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:00.596179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a2219-a121-4216-870e-b40883a2ae89 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.596461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a2219-a121-4216-870e-b40883a2ae89 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.596735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a2219-a121-4216-870e-b40883a2ae89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.609133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a2219-a121-4216-870e-b40883a2ae89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3c8a2219-a121-4216-870e-b40883a2ae89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:00.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:00.619903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:00.620645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:00.624303 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 15/28] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:00.631900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e48da-9034-4ae4-8b38-0a6dafb3fa12 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.632420 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e48da-9034-4ae4-8b38-0a6dafb3fa12 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.632868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e48da-9034-4ae4-8b38-0a6dafb3fa12 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.634827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 16/29] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:00.635884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e48da-9034-4ae4-8b38-0a6dafb3fa12 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:00.663435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff31a-6377-4e80-a7d4-85e983f98dfb None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:00.711027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a27e66-3874-4807-8c07-59371ba3132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:00.712628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a27e66-3874-4807-8c07-59371ba3132d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.712983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a27e66-3874-4807-8c07-59371ba3132d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.713901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a27e66-3874-4807-8c07-59371ba3132d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.729429 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a27e66-3874-4807-8c07-59371ba3132d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:00.731679 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a27e66-3874-4807-8c07-59371ba3132d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:00.763491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e48da-9034-4ae4-8b38-0a6dafb3fa12 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-e3a27e66-3874-4807-8c07-59371ba3132d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:47:00.769071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:00.769885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:00.770879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:47:00.770879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:47:00.770879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:47:00.770879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:00.776523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 14/30] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:00.777471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a27e66-3874-4807-8c07-59371ba3132d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:00.806596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066fd31-0cad-4434-9c29-9f118c0cf809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:00.807631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066fd31-0cad-4434-9c29-9f118c0cf809 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.808107 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066fd31-0cad-4434-9c29-9f118c0cf809 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.808572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066fd31-0cad-4434-9c29-9f118c0cf809 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.809504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066fd31-0cad-4434-9c29-9f118c0cf809 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:00.811225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066fd31-0cad-4434-9c29-9f118c0cf809 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:00.814323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:00.821932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:00.901407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a27e66-3874-4807-8c07-59371ba3132d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:00.940403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c58f4-6808-4bb6-85a0-122d1a9dc760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:00.941213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c58f4-6808-4bb6-85a0-122d1a9dc760 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.941503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c58f4-6808-4bb6-85a0-122d1a9dc760 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.942300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c58f4-6808-4bb6-85a0-122d1a9dc760 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.943457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c58f4-6808-4bb6-85a0-122d1a9dc760 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:00.944243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c58f4-6808-4bb6-85a0-122d1a9dc760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:00.946336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c58f4-6808-4bb6-85a0-122d1a9dc760 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:00.966123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 15/31] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:00.966123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fbe1e-02de-4f56-a082-342ad05e6edd None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:00.966123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fbe1e-02de-4f56-a082-342ad05e6edd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:00.966123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fbe1e-02de-4f56-a082-342ad05e6edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:00.983917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fa3fbe1e-02de-4f56-a082-342ad05e6edd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:00.985673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fa3fbe1e-02de-4f56-a082-342ad05e6edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:01.065291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 16/32] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:01.066416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fbe1e-02de-4f56-a082-342ad05e6edd None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:01.070118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 17/33] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 13 08:47:00 2022] POST /identity/v3/roles => generated 219 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:01.071265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066fd31-0cad-4434-9c29-9f118c0cf809 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:01.115056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:01.275984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066fd31-0cad-4434-9c29-9f118c0cf809 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:01.287340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fbe1e-02de-4f56-a082-342ad05e6edd None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:01.315905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a2ae0-2224-45e9-b44d-4a8a61d96fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:01.316876 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a2ae0-2224-45e9-b44d-4a8a61d96fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:01.317759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a2ae0-2224-45e9-b44d-4a8a61d96fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:01.318083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a2ae0-2224-45e9-b44d-4a8a61d96fe6 None admin] PATH_INFO: `/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:01.319729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a2ae0-2224-45e9-b44d-4a8a61d96fe6 None admin] RBAC: Authorizing `identity:get_user(user_id=26ad26ac4aea4a7b8fb923b47a154aa9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:01.321334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a2ae0-2224-45e9-b44d-4a8a61d96fe6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:01.323310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 18/34] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:01 2022] GET /identity/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9 => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:01.324075 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a2ae0-2224-45e9-b44d-4a8a61d96fe6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:01.333103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34a77c-3e3b-4e16-8ed3-c595d4ac37af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:01.333785 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34a77c-3e3b-4e16-8ed3-c595d4ac37af None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:01.334182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34a77c-3e3b-4e16-8ed3-c595d4ac37af None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:01.334463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34a77c-3e3b-4e16-8ed3-c595d4ac37af None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:01.335200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34a77c-3e3b-4e16-8ed3-c595d4ac37af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:01.336320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34a77c-3e3b-4e16-8ed3-c595d4ac37af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:01.337732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34a77c-3e3b-4e16-8ed3-c595d4ac37af None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:01.357450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 17/35] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:01.358542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34a77c-3e3b-4e16-8ed3-c595d4ac37af None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:01.419226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:01.464941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:01.475340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a2ae0-2224-45e9-b44d-4a8a61d96fe6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:01.488196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34a77c-3e3b-4e16-8ed3-c595d4ac37af None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:01.518738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff45a9-c391-4a7a-881d-de064fe00e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:01.519442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff45a9-c391-4a7a-881d-de064fe00e0c None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:01.519746 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff45a9-c391-4a7a-881d-de064fe00e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:01.520070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff45a9-c391-4a7a-881d-de064fe00e0c None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:01.520837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff45a9-c391-4a7a-881d-de064fe00e0c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:01.523050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff45a9-c391-4a7a-881d-de064fe00e0c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:01.536290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947dc01d-f626-40dc-9868-ce57931d7907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:01.537026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc01d-f626-40dc-9868-ce57931d7907 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:01.537313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc01d-f626-40dc-9868-ce57931d7907 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:01.537579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc01d-f626-40dc-9868-ce57931d7907 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:01.539043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947dc01d-f626-40dc-9868-ce57931d7907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:01.545229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947dc01d-f626-40dc-9868-ce57931d7907 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:01.545229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 18/36] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:01.545229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947dc01d-f626-40dc-9868-ce57931d7907 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:01.593997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 19/37] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 13 08:47:01 2022] POST /identity/v3/roles => generated 225 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:01.595555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ee532-3e2e-4d83-b2c5-924cec59f9e8 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:01.595989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ee532-3e2e-4d83-b2c5-924cec59f9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:01.596341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ee532-3e2e-4d83-b2c5-924cec59f9e8 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:01.597708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 20/38] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:01.598407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ee532-3e2e-4d83-b2c5-924cec59f9e8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:01.661808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947dc01d-f626-40dc-9868-ce57931d7907 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:01.701493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06107825-2ea2-443b-a193-b7ee7c271a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:01.702373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107825-2ea2-443b-a193-b7ee7c271a19 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:01.702662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107825-2ea2-443b-a193-b7ee7c271a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:01.702939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107825-2ea2-443b-a193-b7ee7c271a19 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:01.703663 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06107825-2ea2-443b-a193-b7ee7c271a19 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:01.705113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06107825-2ea2-443b-a193-b7ee7c271a19 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:01.721711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ee532-3e2e-4d83-b2c5-924cec59f9e8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:01.749200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:01.761447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd007c09-f476-4560-b976-25fb33d643de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:01.762435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007c09-f476-4560-b976-25fb33d643de None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:01.763448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007c09-f476-4560-b976-25fb33d643de None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:01.763448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007c09-f476-4560-b976-25fb33d643de None admin] PATH_INFO: `/v3/domains/default/users/26ad26ac4aea4a7b8fb923b47a154aa9/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:01.765279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd007c09-f476-4560-b976-25fb33d643de None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=26ad26ac4aea4a7b8fb923b47a154aa9, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:01.766803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd007c09-f476-4560-b976-25fb33d643de None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:01.833195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:01.858476 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 21/39] 10.222.0.40 () {58 vars in 1361 bytes} [Wed Jul 13 08:47:01 2022] PUT /identity/v3/domains/default/users/26ad26ac4aea4a7b8fb923b47a154aa9/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:01.862474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986007-0bf2-4379-91db-a4f3ce5b5f5c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:01.862474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986007-0bf2-4379-91db-a4f3ce5b5f5c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:01.862474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986007-0bf2-4379-91db-a4f3ce5b5f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:01.878214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7c986007-0bf2-4379-91db-a4f3ce5b5f5c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:01.880364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7c986007-0bf2-4379-91db-a4f3ce5b5f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:01.950226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 19/40] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:47:01 2022] POST /identity/v3/projects => generated 301 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:01.950226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1aca-bddd-47e2-bc7c-212038be46c8 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:01.950226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1aca-bddd-47e2-bc7c-212038be46c8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:01.950226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1aca-bddd-47e2-bc7c-212038be46c8 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:01.950226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 20/41] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:01.958143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003f399-e269-4ada-a081-1d94d81a43c9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:01.958143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003f399-e269-4ada-a081-1d94d81a43c9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:01.958143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003f399-e269-4ada-a081-1d94d81a43c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:01.976242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d003f399-e269-4ada-a081-1d94d81a43c9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:01.977157 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 22/42] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:01.978689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d003f399-e269-4ada-a081-1d94d81a43c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:02.015828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c986007-0bf2-4379-91db-a4f3ce5b5f5c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:02.073903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 21/43] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:02.103943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003f399-e269-4ada-a081-1d94d81a43c9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:02.144791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c986007-0bf2-4379-91db-a4f3ce5b5f5c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:02.159642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:02.187887 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f6fb7-cb40-4c33-81f9-5403d0d6d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:02.188623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f6fb7-cb40-4c33-81f9-5403d0d6d57e None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:02.189107 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f6fb7-cb40-4c33-81f9-5403d0d6d57e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:02.189548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f6fb7-cb40-4c33-81f9-5403d0d6d57e None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:02.190709 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f6fb7-cb40-4c33-81f9-5403d0d6d57e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:02.192503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f6fb7-cb40-4c33-81f9-5403d0d6d57e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:02.230647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003f399-e269-4ada-a081-1d94d81a43c9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:02.260902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:02.266648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c5d68-e50a-4b23-b324-f9057993fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:02.267461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c5d68-e50a-4b23-b324-f9057993fd8d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:02.267737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c5d68-e50a-4b23-b324-f9057993fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:02.268016 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c5d68-e50a-4b23-b324-f9057993fd8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:02.269234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c5d68-e50a-4b23-b324-f9057993fd8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:02.271116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c5d68-e50a-4b23-b324-f9057993fd8d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:02.272905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 22/44] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:02.279028 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c5d68-e50a-4b23-b324-f9057993fd8d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:02.303027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 23/45] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 13 08:47:02 2022] POST /identity/v3/roles => generated 223 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:47:02.504393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c5d68-e50a-4b23-b324-f9057993fd8d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:02.539820 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be2526-6456-442a-a965-5d915aefd749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:02.540486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be2526-6456-442a-a965-5d915aefd749 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:02.540772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be2526-6456-442a-a965-5d915aefd749 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:02.541050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be2526-6456-442a-a965-5d915aefd749 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:02.541757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be2526-6456-442a-a965-5d915aefd749 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:02.543251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be2526-6456-442a-a965-5d915aefd749 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:02.616317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 23/46] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:47:02 2022] POST /identity/v3/projects => generated 312 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:47:03.946712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3688f2-b5fc-4dda-aa23-3f961fb07df5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:03.947233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3688f2-b5fc-4dda-aa23-3f961fb07df5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:03.947315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3688f2-b5fc-4dda-aa23-3f961fb07df5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:03.948647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 24/47] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:03.954001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cebbb6-3563-4855-a6ce-8ed109fedef8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:03.954411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cebbb6-3563-4855-a6ce-8ed109fedef8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:03.954585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cebbb6-3563-4855-a6ce-8ed109fedef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:03.971284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-39cebbb6-3563-4855-a6ce-8ed109fedef8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:03.972455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-39cebbb6-3563-4855-a6ce-8ed109fedef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:04.018601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:04.039718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 24/48] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:04.063332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3688f2-b5fc-4dda-aa23-3f961fb07df5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:04.164935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3688f2-b5fc-4dda-aa23-3f961fb07df5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:04.198093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7336ef3-866b-4f90-970f-f5188dba16bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:04.198803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7336ef3-866b-4f90-970f-f5188dba16bf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:04.199087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7336ef3-866b-4f90-970f-f5188dba16bf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:04.199363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7336ef3-866b-4f90-970f-f5188dba16bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:04.209429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7336ef3-866b-4f90-970f-f5188dba16bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b7336ef3-866b-4f90-970f-f5188dba16bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:04.217032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:04.217848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:04.218635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 25/49] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:04.224634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cebbb6-3563-4855-a6ce-8ed109fedef8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:04.323406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cebbb6-3563-4855-a6ce-8ed109fedef8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:04.358520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2900f-10b9-4ae3-ab28-dbceb9cd9d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:04.359187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2900f-10b9-4ae3-ab28-dbceb9cd9d1d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:04.359476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2900f-10b9-4ae3-ab28-dbceb9cd9d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:04.359753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2900f-10b9-4ae3-ab28-dbceb9cd9d1d None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:04.360500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2900f-10b9-4ae3-ab28-dbceb9cd9d1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:04.360805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2900f-10b9-4ae3-ab28-dbceb9cd9d1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:04.362205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2900f-10b9-4ae3-ab28-dbceb9cd9d1d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:04.371707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 25/50] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:04.377117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7336ef3-866b-4f90-970f-f5188dba16bf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:04.513853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:04.583072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7336ef3-866b-4f90-970f-f5188dba16bf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:04.621097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d95661-5671-4f15-88bd-37d38f1e7f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:04.622000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d95661-5671-4f15-88bd-37d38f1e7f8a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:04.622353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d95661-5671-4f15-88bd-37d38f1e7f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:04.622637 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d95661-5671-4f15-88bd-37d38f1e7f8a None admin] PATH_INFO: `/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:04.623873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d95661-5671-4f15-88bd-37d38f1e7f8a None admin] RBAC: Authorizing `identity:get_user(user_id=26ad26ac4aea4a7b8fb923b47a154aa9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:04.625326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d95661-5671-4f15-88bd-37d38f1e7f8a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:04.626982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 26/51] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:04 2022] GET /identity/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9 => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:04.632985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2900f-10b9-4ae3-ab28-dbceb9cd9d1d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:04.663941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfab0b8-ae4e-484c-9f2c-6eab4db8a0a2 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:04.664337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfab0b8-ae4e-484c-9f2c-6eab4db8a0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:04.664674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfab0b8-ae4e-484c-9f2c-6eab4db8a0a2 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:04.666291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 27/52] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:47:04.671491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17d568-ca87-4d7d-bf84-5cbe77f4aa66 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:04.672033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17d568-ca87-4d7d-bf84-5cbe77f4aa66 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:04.672317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17d568-ca87-4d7d-bf84-5cbe77f4aa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:04.684791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:04.690259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bb17d568-ca87-4d7d-bf84-5cbe77f4aa66 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:04.691457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bb17d568-ca87-4d7d-bf84-5cbe77f4aa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:04.743560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2900f-10b9-4ae3-ab28-dbceb9cd9d1d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:04.758832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 28/53] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:04.776804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ca3c7-d661-49b2-b527-5f2f37855c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:04.777534 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca3c7-d661-49b2-b527-5f2f37855c04 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:04.777815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca3c7-d661-49b2-b527-5f2f37855c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:04.778124 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca3c7-d661-49b2-b527-5f2f37855c04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:04.779302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ca3c7-d661-49b2-b527-5f2f37855c04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:04.780989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ca3c7-d661-49b2-b527-5f2f37855c04 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:04.781913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17d568-ca87-4d7d-bf84-5cbe77f4aa66 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:04.782375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 26/54] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:04.787890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ca3c7-d661-49b2-b527-5f2f37855c04 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:04.811907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:04.882198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17d568-ca87-4d7d-bf84-5cbe77f4aa66 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:04.886994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ca3c7-d661-49b2-b527-5f2f37855c04 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:04.898297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:04.918596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8a2ba-8768-4ea7-a3a9-ddf7c6a4969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:04.919376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8a2ba-8768-4ea7-a3a9-ddf7c6a4969f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:04.919747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8a2ba-8768-4ea7-a3a9-ddf7c6a4969f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:04.920026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8a2ba-8768-4ea7-a3a9-ddf7c6a4969f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:04.921204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75ba3-6f37-419c-9c8a-f1173e0c8442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:04.921312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8a2ba-8768-4ea7-a3a9-ddf7c6a4969f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:04.922146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75ba3-6f37-419c-9c8a-f1173e0c8442 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:04.922431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75ba3-6f37-419c-9c8a-f1173e0c8442 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:04.922707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75ba3-6f37-419c-9c8a-f1173e0c8442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:04.922970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8a2ba-8768-4ea7-a3a9-ddf7c6a4969f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:04.923532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e75ba3-6f37-419c-9c8a-f1173e0c8442 None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.domain.id=default, user.id=26ad26ac4aea4a7b8fb923b47a154aa9) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:04.923842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e75ba3-6f37-419c-9c8a-f1173e0c8442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:04.924568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 29/55] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:04.925330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e75ba3-6f37-419c-9c8a-f1173e0c8442 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:04.931167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8a2ba-8768-4ea7-a3a9-ddf7c6a4969f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:04.934813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 27/56] 10.222.0.40 () {56 vars in 1322 bytes} [Wed Jul 13 08:47:04 2022] GET /identity/v3/role_assignments?user.id=26ad26ac4aea4a7b8fb923b47a154aa9&scope.domain.id=default&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:05.034677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:05.039114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8a2ba-8768-4ea7-a3a9-ddf7c6a4969f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:05.077831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d84f4d-c57d-4080-aebe-221563847650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:05.078528 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d84f4d-c57d-4080-aebe-221563847650 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:05.078816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d84f4d-c57d-4080-aebe-221563847650 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:05.079104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d84f4d-c57d-4080-aebe-221563847650 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:05.079877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d84f4d-c57d-4080-aebe-221563847650 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:05.081337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d84f4d-c57d-4080-aebe-221563847650 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:05.182322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 30/57] 10.222.0.40 () {60 vars in 1140 bytes} [Wed Jul 13 08:47:04 2022] POST /identity/v3/projects => generated 298 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:47:07.176253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28fbd0-e683-412a-aef5-9b5f7b232eb2 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:07.176809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28fbd0-e683-412a-aef5-9b5f7b232eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:07.177063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28fbd0-e683-412a-aef5-9b5f7b232eb2 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:07.178472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 28/58] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:07.183704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80cc53-2239-479d-b0b0-73280735fca6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:07.184100 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80cc53-2239-479d-b0b0-73280735fca6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:07.184261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80cc53-2239-479d-b0b0-73280735fca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:07.201303 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5b80cc53-2239-479d-b0b0-73280735fca6 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:07.202510 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5b80cc53-2239-479d-b0b0-73280735fca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:07.265636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 31/59] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:07.290146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28fbd0-e683-412a-aef5-9b5f7b232eb2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:07.391051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28fbd0-e683-412a-aef5-9b5f7b232eb2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:07.423053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f7f6a-276b-4180-ac0b-b0b371af6ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:07.423834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f7f6a-276b-4180-ac0b-b0b371af6ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:07.424116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f7f6a-276b-4180-ac0b-b0b371af6ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:07.424393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f7f6a-276b-4180-ac0b-b0b371af6ac1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:07.425551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f7f6a-276b-4180-ac0b-b0b371af6ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:07.427559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f7f6a-276b-4180-ac0b-b0b371af6ac1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:07.429081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 29/60] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:07.434570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80cc53-2239-479d-b0b0-73280735fca6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:07.513420 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:07.531804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80cc53-2239-479d-b0b0-73280735fca6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:07.565017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7540db3-8136-4dd7-a3ba-3d5d302c1dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:07.565673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7540db3-8136-4dd7-a3ba-3d5d302c1dc4 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:07.565960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7540db3-8136-4dd7-a3ba-3d5d302c1dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:07.566267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7540db3-8136-4dd7-a3ba-3d5d302c1dc4 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:07.567626 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7540db3-8136-4dd7-a3ba-3d5d302c1dc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:07.569390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7540db3-8136-4dd7-a3ba-3d5d302c1dc4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:07.651549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 32/61] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:47:07 2022] POST /identity/v3/users => generated 275 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:10.660032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cca617-1722-4ce4-b6a7-b973ff69d834 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:10.661654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cca617-1722-4ce4-b6a7-b973ff69d834 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:10.662165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cca617-1722-4ce4-b6a7-b973ff69d834 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:10.663744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 30/62] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:10.669155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130fa19-4ea6-45b9-89e6-f1b29b552257 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:10.669960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130fa19-4ea6-45b9-89e6-f1b29b552257 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:10.670562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130fa19-4ea6-45b9-89e6-f1b29b552257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:10.688997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5130fa19-4ea6-45b9-89e6-f1b29b552257 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:10.690526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5130fa19-4ea6-45b9-89e6-f1b29b552257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:10.761858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 33/63] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:10.786241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cca617-1722-4ce4-b6a7-b973ff69d834 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:10.837899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:10.880739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29340d-f56d-4ad6-86ae-42cc55a9c8f6 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:10.881378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29340d-f56d-4ad6-86ae-42cc55a9c8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:10.881857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29340d-f56d-4ad6-86ae-42cc55a9c8f6 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:10.883930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 34/64] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:47:10.894090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8100f7-4e61-4607-a21b-196dd62073af None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:10.894811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8100f7-4e61-4607-a21b-196dd62073af None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:10.895791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8100f7-4e61-4607-a21b-196dd62073af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:10.919223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-dd8100f7-4e61-4607-a21b-196dd62073af None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:10.920361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dd8100f7-4e61-4607-a21b-196dd62073af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:10.937363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cca617-1722-4ce4-b6a7-b973ff69d834 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:11.002535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2289a-290d-4036-9beb-1f642032a102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:11.003357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2289a-290d-4036-9beb-1f642032a102 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.003642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2289a-290d-4036-9beb-1f642032a102 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.003921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2289a-290d-4036-9beb-1f642032a102 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.028531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2289a-290d-4036-9beb-1f642032a102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-8ce2289a-290d-4036-9beb-1f642032a102 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:11.043007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:11.043865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:11.044622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 31/65] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:11.054940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:11.055989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2289a-290d-4036-9beb-1f642032a102 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:11.057811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 35/66] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 13 08:47:11.081182 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8100f7-4e61-4607-a21b-196dd62073af None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:11.213062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8100f7-4e61-4607-a21b-196dd62073af None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:11.255436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b4a29-638a-463f-bb33-757534b8d021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:11.256545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b4a29-638a-463f-bb33-757534b8d021 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.256834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b4a29-638a-463f-bb33-757534b8d021 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.257119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b4a29-638a-463f-bb33-757534b8d021 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.264371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2289a-290d-4036-9beb-1f642032a102 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:11.266436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:11.269501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b4a29-638a-463f-bb33-757534b8d021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-420b4a29-638a-463f-bb33-757534b8d021 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:11.279020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:11.281366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 36/67] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:11.282452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf99d3-40b7-4ce6-b760-396060159e1f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.283152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf99d3-40b7-4ce6-b760-396060159e1f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.283152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf99d3-40b7-4ce6-b760-396060159e1f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.284368 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 37/68] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:11.289052 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf99d3-40b7-4ce6-b760-396060159e1f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:11.313284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae06c0-c4eb-4f09-9482-d03629fa148d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:11.314221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae06c0-c4eb-4f09-9482-d03629fa148d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.314645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae06c0-c4eb-4f09-9482-d03629fa148d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.315063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae06c0-c4eb-4f09-9482-d03629fa148d None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.315975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae06c0-c4eb-4f09-9482-d03629fa148d None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:11.316423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae06c0-c4eb-4f09-9482-d03629fa148d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:11.318737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae06c0-c4eb-4f09-9482-d03629fa148d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:11.327693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 32/69] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:11.329257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135bf22-5484-4fd9-9746-4007f6a41e31 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.329718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135bf22-5484-4fd9-9746-4007f6a41e31 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.330216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135bf22-5484-4fd9-9746-4007f6a41e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.352797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0135bf22-5484-4fd9-9746-4007f6a41e31 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:11.354261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0135bf22-5484-4fd9-9746-4007f6a41e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:11.365227 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:11.409852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf99d3-40b7-4ce6-b760-396060159e1f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:11.422554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 33/70] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:11.423583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135bf22-5484-4fd9-9746-4007f6a41e31 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:11.458470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331e7b9-bebd-470e-8886-e80e9467cd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:11.459305 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331e7b9-bebd-470e-8886-e80e9467cd79 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.459658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331e7b9-bebd-470e-8886-e80e9467cd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.460008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331e7b9-bebd-470e-8886-e80e9467cd79 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.460818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331e7b9-bebd-470e-8886-e80e9467cd79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:11.461193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331e7b9-bebd-470e-8886-e80e9467cd79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:11.462688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331e7b9-bebd-470e-8886-e80e9467cd79 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:11.470881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 38/71] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:11.472030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c434491-b0f2-484c-91b4-15b31910eafd None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.472312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c434491-b0f2-484c-91b4-15b31910eafd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.472591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c434491-b0f2-484c-91b4-15b31910eafd None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.474052 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 39/72] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:11.475070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37085-2c8d-4cb3-94a3-f278204bf2a5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.475591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37085-2c8d-4cb3-94a3-f278204bf2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.475874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37085-2c8d-4cb3-94a3-f278204bf2a5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.477078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 40/73] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:11.477821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a37085-2c8d-4cb3-94a3-f278204bf2a5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:11.523413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:11.529795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135bf22-5484-4fd9-9746-4007f6a41e31 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:11.563378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa28765-e1e2-4001-ba5d-60fea72f527d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:11.564450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa28765-e1e2-4001-ba5d-60fea72f527d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.564738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa28765-e1e2-4001-ba5d-60fea72f527d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.565018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa28765-e1e2-4001-ba5d-60fea72f527d None admin] PATH_INFO: `/v3/users/654695f326044efdb73fe2956b7ce735` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.579764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a37085-2c8d-4cb3-94a3-f278204bf2a5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:11.580643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa28765-e1e2-4001-ba5d-60fea72f527d None admin] RBAC: Authorizing `identity:get_user(user_id=654695f326044efdb73fe2956b7ce735)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:11.582167 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa28765-e1e2-4001-ba5d-60fea72f527d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:11.584509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 34/74] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:11 2022] GET /identity/v3/users/654695f326044efdb73fe2956b7ce735 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:11.585276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa28765-e1e2-4001-ba5d-60fea72f527d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:11.599679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:11.615829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efc2c2-7aac-4a2d-913f-e5b7bb8b5633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:11.616493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efc2c2-7aac-4a2d-913f-e5b7bb8b5633 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.616778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efc2c2-7aac-4a2d-913f-e5b7bb8b5633 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.617058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efc2c2-7aac-4a2d-913f-e5b7bb8b5633 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.627548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efc2c2-7aac-4a2d-913f-e5b7bb8b5633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-11efc2c2-7aac-4a2d-913f-e5b7bb8b5633 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:11.636429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:11.637299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 41/75] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:11.642942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cedd2-c0ba-42ee-8699-6e2678ba90c0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.643715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cedd2-c0ba-42ee-8699-6e2678ba90c0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.643715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cedd2-c0ba-42ee-8699-6e2678ba90c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.656101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e62cedd2-c0ba-42ee-8699-6e2678ba90c0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:11.657292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e62cedd2-c0ba-42ee-8699-6e2678ba90c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:11.689651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa28765-e1e2-4001-ba5d-60fea72f527d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:11.722848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 42/76] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:11.724220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43789b-3b8c-4f6c-ae18-d11244443801 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.724513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43789b-3b8c-4f6c-ae18-d11244443801 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.724791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43789b-3b8c-4f6c-ae18-d11244443801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.728627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6514f-4c22-41d4-8554-e6a774295a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:11.729692 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6514f-4c22-41d4-8554-e6a774295a93 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.730783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6514f-4c22-41d4-8554-e6a774295a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.731079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6514f-4c22-41d4-8554-e6a774295a93 None admin] PATH_INFO: `/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.732332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6514f-4c22-41d4-8554-e6a774295a93 None admin] RBAC: Authorizing `identity:get_user(user_id=26ad26ac4aea4a7b8fb923b47a154aa9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:11.733792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6514f-4c22-41d4-8554-e6a774295a93 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:11.740693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 35/77] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:11 2022] GET /identity/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:11.740693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6514f-4c22-41d4-8554-e6a774295a93 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:11.740881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7e43789b-3b8c-4f6c-ae18-d11244443801 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:11.741542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7e43789b-3b8c-4f6c-ae18-d11244443801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:11.760463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:11.809615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 43/78] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:11.810487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43789b-3b8c-4f6c-ae18-d11244443801 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:11.842879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6514f-4c22-41d4-8554-e6a774295a93 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:11.852999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:11.876592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dc6fa-617e-4ee0-b7a4-0049ee9f69ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:11.877251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dc6fa-617e-4ee0-b7a4-0049ee9f69ce None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.877537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dc6fa-617e-4ee0-b7a4-0049ee9f69ce None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.877816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dc6fa-617e-4ee0-b7a4-0049ee9f69ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.911287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43789b-3b8c-4f6c-ae18-d11244443801 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:11.944813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230464f0-383b-404f-8de2-3b9864d2071c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:11.945475 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230464f0-383b-404f-8de2-3b9864d2071c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:11.945753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230464f0-383b-404f-8de2-3b9864d2071c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:11.946011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dc6fa-617e-4ee0-b7a4-0049ee9f69ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:11.946124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230464f0-383b-404f-8de2-3b9864d2071c None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:11.946784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230464f0-383b-404f-8de2-3b9864d2071c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:11.947091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230464f0-383b-404f-8de2-3b9864d2071c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:11.947525 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dc6fa-617e-4ee0-b7a4-0049ee9f69ce None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:11.948467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230464f0-383b-404f-8de2-3b9864d2071c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:11.956207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 44/79] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:11.957056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230464f0-383b-404f-8de2-3b9864d2071c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:11.970229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:12.050481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 36/80] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:12.051379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dc6fa-617e-4ee0-b7a4-0049ee9f69ce None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:12.054786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:12.057535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230464f0-383b-404f-8de2-3b9864d2071c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:12.092563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322a94a-b3c3-4c94-99be-ea3be7604d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:12.093604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322a94a-b3c3-4c94-99be-ea3be7604d0a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:12.093891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322a94a-b3c3-4c94-99be-ea3be7604d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:12.094199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322a94a-b3c3-4c94-99be-ea3be7604d0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:12.152135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dc6fa-617e-4ee0-b7a4-0049ee9f69ce None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:12.161065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322a94a-b3c3-4c94-99be-ea3be7604d0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:12.162582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322a94a-b3c3-4c94-99be-ea3be7604d0a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:12.172292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:12.185779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd30bc-ba2e-4ce7-8dca-77144fc7eb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:12.186600 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd30bc-ba2e-4ce7-8dca-77144fc7eb8b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:12.186881 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd30bc-ba2e-4ce7-8dca-77144fc7eb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:12.187156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd30bc-ba2e-4ce7-8dca-77144fc7eb8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:12.198274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd30bc-ba2e-4ce7-8dca-77144fc7eb8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-75dd30bc-ba2e-4ce7-8dca-77144fc7eb8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:12.206486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:12.207973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:12.208732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 37/81] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:12.209402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd30bc-ba2e-4ce7-8dca-77144fc7eb8b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:12.259872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:12.263580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 45/82] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:12.264377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322a94a-b3c3-4c94-99be-ea3be7604d0a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:12.307045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd30bc-ba2e-4ce7-8dca-77144fc7eb8b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:12.340760 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e8ec6-71ad-4ef5-baf4-e2763e42c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:12.341563 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e8ec6-71ad-4ef5-baf4-e2763e42c13b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:12.341839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e8ec6-71ad-4ef5-baf4-e2763e42c13b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:12.342144 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e8ec6-71ad-4ef5-baf4-e2763e42c13b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:12.343314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e8ec6-71ad-4ef5-baf4-e2763e42c13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:12.344752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e8ec6-71ad-4ef5-baf4-e2763e42c13b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:12.346368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 38/83] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:12.347119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e8ec6-71ad-4ef5-baf4-e2763e42c13b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:12.365871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322a94a-b3c3-4c94-99be-ea3be7604d0a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:12.397123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:12.399797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbb136-386f-41f9-8105-011c126a3d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:12.400662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbb136-386f-41f9-8105-011c126a3d0a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:12.400957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbb136-386f-41f9-8105-011c126a3d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:12.401234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbb136-386f-41f9-8105-011c126a3d0a None admin] PATH_INFO: `/v3/users/654695f326044efdb73fe2956b7ce735` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:12.402459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbb136-386f-41f9-8105-011c126a3d0a None admin] RBAC: Authorizing `identity:get_user(user_id=654695f326044efdb73fe2956b7ce735)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:12.403921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbb136-386f-41f9-8105-011c126a3d0a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:12.405531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 46/84] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:12 2022] GET /identity/v3/users/654695f326044efdb73fe2956b7ce735 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:12.406309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbb136-386f-41f9-8105-011c126a3d0a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:12.449547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e8ec6-71ad-4ef5-baf4-e2763e42c13b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:12.471408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:12.482983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901c352-45c5-4a48-9a2f-63915f1abc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:12.483839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901c352-45c5-4a48-9a2f-63915f1abc03 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:12.484119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901c352-45c5-4a48-9a2f-63915f1abc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:12.484397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901c352-45c5-4a48-9a2f-63915f1abc03 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:12.485603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901c352-45c5-4a48-9a2f-63915f1abc03 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:12.487090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901c352-45c5-4a48-9a2f-63915f1abc03 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:12.489432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 39/85] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:12 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:12.490214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901c352-45c5-4a48-9a2f-63915f1abc03 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:12.504469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbb136-386f-41f9-8105-011c126a3d0a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:12.539116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da426f24-b723-47e1-b1ee-497a22557c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:12.539791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da426f24-b723-47e1-b1ee-497a22557c61 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:12.540079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da426f24-b723-47e1-b1ee-497a22557c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:12.540362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da426f24-b723-47e1-b1ee-497a22557c61 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:12.541114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da426f24-b723-47e1-b1ee-497a22557c61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:12.541424 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da426f24-b723-47e1-b1ee-497a22557c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:12.542847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da426f24-b723-47e1-b1ee-497a22557c61 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:12.551210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 47/86] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:12.551962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da426f24-b723-47e1-b1ee-497a22557c61 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:12.592481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901c352-45c5-4a48-9a2f-63915f1abc03 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:12.608482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:12.626264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e092c-3884-476b-8a86-8fc8de517b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:12.627118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e092c-3884-476b-8a86-8fc8de517b70 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:12.627396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e092c-3884-476b-8a86-8fc8de517b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:12.627668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e092c-3884-476b-8a86-8fc8de517b70 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:12.628867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e092c-3884-476b-8a86-8fc8de517b70 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:12.630611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e092c-3884-476b-8a86-8fc8de517b70 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:12.632287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 40/87] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:12 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:12.633039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e092c-3884-476b-8a86-8fc8de517b70 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:12.650643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da426f24-b723-47e1-b1ee-497a22557c61 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:12.679678 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:12.685251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7180d6-e384-4faa-9ae5-4b7e48c609b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:12.685912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7180d6-e384-4faa-9ae5-4b7e48c609b8 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:12.686228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7180d6-e384-4faa-9ae5-4b7e48c609b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:12.686515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7180d6-e384-4faa-9ae5-4b7e48c609b8 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:12.691029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7180d6-e384-4faa-9ae5-4b7e48c609b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:12.691029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7180d6-e384-4faa-9ae5-4b7e48c609b8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:12.737076 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e092c-3884-476b-8a86-8fc8de517b70 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:12.778001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2545631-8120-4113-9922-b2b0caae3016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:12.778673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2545631-8120-4113-9922-b2b0caae3016 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:12.778962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2545631-8120-4113-9922-b2b0caae3016 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:12.779241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2545631-8120-4113-9922-b2b0caae3016 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:12.800651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 48/88] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:47:12 2022] POST /identity/v3/users => generated 289 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:12.801641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7180d6-e384-4faa-9ae5-4b7e48c609b8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:12.837084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:12.856128 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2545631-8120-4113-9922-b2b0caae3016 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:12.857719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2545631-8120-4113-9922-b2b0caae3016 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:12.922861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7180d6-e384-4faa-9ae5-4b7e48c609b8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:12.962473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda7da4-5a8d-452d-b869-42d8ce0527ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:12.963229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda7da4-5a8d-452d-b869-42d8ce0527ad None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:12.964114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda7da4-5a8d-452d-b869-42d8ce0527ad None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:12.964114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda7da4-5a8d-452d-b869-42d8ce0527ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:12.964635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda7da4-5a8d-452d-b869-42d8ce0527ad None admin] RBAC: Adding query filter params (role.id=a70b63ede7014fa9bbcb4e03275a2664, scope.project.id=245d48325d884a83b0a57436df3e1418, user.id=654695f326044efdb73fe2956b7ce735) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:12.964964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda7da4-5a8d-452d-b869-42d8ce0527ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:12.966535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda7da4-5a8d-452d-b869-42d8ce0527ad None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:12.972637 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 49/89] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:12 2022] GET /identity/v3/role_assignments?user.id=654695f326044efdb73fe2956b7ce735&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:12.974520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda7da4-5a8d-452d-b869-42d8ce0527ad None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:12.976961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 41/90] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:12.978093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2545631-8120-4113-9922-b2b0caae3016 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:13.056884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:13.129482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:13.149519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2545631-8120-4113-9922-b2b0caae3016 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:13.153258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda7da4-5a8d-452d-b869-42d8ce0527ad None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:13.196483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525794f-71e7-4eb1-869b-657a1ece10aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:13.197377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525794f-71e7-4eb1-869b-657a1ece10aa None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:13.197657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525794f-71e7-4eb1-869b-657a1ece10aa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:13.197938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525794f-71e7-4eb1-869b-657a1ece10aa None admin] PATH_INFO: `/v3/users/654695f326044efdb73fe2956b7ce735` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:13.199188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525794f-71e7-4eb1-869b-657a1ece10aa None admin] RBAC: Authorizing `identity:get_user(user_id=654695f326044efdb73fe2956b7ce735)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:13.200677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525794f-71e7-4eb1-869b-657a1ece10aa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:13.202499 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 50/91] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:12 2022] GET /identity/v3/users/654695f326044efdb73fe2956b7ce735 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:13.203101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525794f-71e7-4eb1-869b-657a1ece10aa None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:13.204342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c25a0-8957-40e3-ac1d-4c6dfbfaa352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:13.205203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c25a0-8957-40e3-ac1d-4c6dfbfaa352 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:13.205994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c25a0-8957-40e3-ac1d-4c6dfbfaa352 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:13.206479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c25a0-8957-40e3-ac1d-4c6dfbfaa352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:13.208994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c25a0-8957-40e3-ac1d-4c6dfbfaa352 None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=245d48325d884a83b0a57436df3e1418, user.id=26ad26ac4aea4a7b8fb923b47a154aa9) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:13.209529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c25a0-8957-40e3-ac1d-4c6dfbfaa352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:13.211201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c25a0-8957-40e3-ac1d-4c6dfbfaa352 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:13.219578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 42/92] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:12 2022] GET /identity/v3/role_assignments?user.id=26ad26ac4aea4a7b8fb923b47a154aa9&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:13.220827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c25a0-8957-40e3-ac1d-4c6dfbfaa352 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:13.347519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525794f-71e7-4eb1-869b-657a1ece10aa None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:13.361511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c25a0-8957-40e3-ac1d-4c6dfbfaa352 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:13.371883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:13.396470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22393-3b46-48b2-879e-17b433ea2181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:13.397257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22393-3b46-48b2-879e-17b433ea2181 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:13.397547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22393-3b46-48b2-879e-17b433ea2181 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:13.397817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22393-3b46-48b2-879e-17b433ea2181 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:13.399951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22393-3b46-48b2-879e-17b433ea2181 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:13.401992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22393-3b46-48b2-879e-17b433ea2181 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:13.404359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 51/93] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:13 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:13.405313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f1f48-fa64-4872-9fec-2007d00c348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:13.406245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f1f48-fa64-4872-9fec-2007d00c348b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:13.406726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f1f48-fa64-4872-9fec-2007d00c348b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:13.407190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f1f48-fa64-4872-9fec-2007d00c348b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:13.411992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22393-3b46-48b2-879e-17b433ea2181 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:13.429745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:13.530621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f1f48-fa64-4872-9fec-2007d00c348b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:13.533278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f1f48-fa64-4872-9fec-2007d00c348b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:13.584853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22393-3b46-48b2-879e-17b433ea2181 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:13.624976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2c0d3-5ab4-4421-bdf8-53788dbab93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:13.626272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2c0d3-5ab4-4421-bdf8-53788dbab93b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:13.627171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2c0d3-5ab4-4421-bdf8-53788dbab93b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:13.627698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2c0d3-5ab4-4421-bdf8-53788dbab93b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:13.628733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2c0d3-5ab4-4421-bdf8-53788dbab93b None admin] RBAC: Adding query filter params (role.id=120eeffad7094e10a69166d19aaa6439, scope.project.id=245d48325d884a83b0a57436df3e1418, user.id=654695f326044efdb73fe2956b7ce735) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:13.629815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2c0d3-5ab4-4421-bdf8-53788dbab93b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:13.632271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2c0d3-5ab4-4421-bdf8-53788dbab93b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:13.639151 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 52/94] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:13 2022] GET /identity/v3/role_assignments?user.id=654695f326044efdb73fe2956b7ce735&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=120eeffad7094e10a69166d19aaa6439 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:13.725034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 43/95] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:13.730250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2c0d3-5ab4-4421-bdf8-53788dbab93b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:13.806616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:13.877246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2c0d3-5ab4-4421-bdf8-53788dbab93b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:13.928601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f966ef8-7d47-4a02-987f-8b3513cff451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:13.929498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966ef8-7d47-4a02-987f-8b3513cff451 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:13.929787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966ef8-7d47-4a02-987f-8b3513cff451 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:13.930097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966ef8-7d47-4a02-987f-8b3513cff451 None admin] PATH_INFO: `/v3/projects/94ce177c584046218878ea073342cd16` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:13.931379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f966ef8-7d47-4a02-987f-8b3513cff451 None admin] RBAC: Authorizing `identity:get_project(project_id=94ce177c584046218878ea073342cd16)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:13.932881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f966ef8-7d47-4a02-987f-8b3513cff451 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:13.934868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 53/96] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:13 2022] GET /identity/v3/projects/94ce177c584046218878ea073342cd16 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:13.940897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f1f48-fa64-4872-9fec-2007d00c348b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:14.030112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:14.109836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f1f48-fa64-4872-9fec-2007d00c348b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:14.155386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89993cf0-22d7-4fcd-acf4-95bf318042b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:14.156053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89993cf0-22d7-4fcd-acf4-95bf318042b1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:14.156339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89993cf0-22d7-4fcd-acf4-95bf318042b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:14.156616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89993cf0-22d7-4fcd-acf4-95bf318042b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:14.157450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89993cf0-22d7-4fcd-acf4-95bf318042b1 None admin] RBAC: Adding query filter params (role.id=a70b63ede7014fa9bbcb4e03275a2664, scope.project.id=94ce177c584046218878ea073342cd16, user.id=654695f326044efdb73fe2956b7ce735) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:14.157761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89993cf0-22d7-4fcd-acf4-95bf318042b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:14.159227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89993cf0-22d7-4fcd-acf4-95bf318042b1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:14.165588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 44/97] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:13 2022] GET /identity/v3/role_assignments?user.id=654695f326044efdb73fe2956b7ce735&scope.project.id=94ce177c584046218878ea073342cd16&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:15.542512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581677e-5b2b-432f-b1aa-8791aeadaee3 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:15.543008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581677e-5b2b-432f-b1aa-8791aeadaee3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:15.543078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581677e-5b2b-432f-b1aa-8791aeadaee3 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:15.544607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 54/98] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:15.549946 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff883a-e369-4b30-909b-ebad48e1ff70 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:15.550396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff883a-e369-4b30-909b-ebad48e1ff70 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:15.550561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff883a-e369-4b30-909b-ebad48e1ff70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:15.571243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-26ff883a-e369-4b30-909b-ebad48e1ff70 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:15.572446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-26ff883a-e369-4b30-909b-ebad48e1ff70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:15.668505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 45/99] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:15.705754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581677e-5b2b-432f-b1aa-8791aeadaee3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:15.843764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581677e-5b2b-432f-b1aa-8791aeadaee3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:15.866467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:15.891261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04c454-e132-49bc-a78e-246f9438a3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:15.892077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04c454-e132-49bc-a78e-246f9438a3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:15.892370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04c454-e132-49bc-a78e-246f9438a3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:15.892654 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04c454-e132-49bc-a78e-246f9438a3f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:15.911643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04c454-e132-49bc-a78e-246f9438a3f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-db04c454-e132-49bc-a78e-246f9438a3f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:15.919441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:15.920272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:15.921022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 55/100] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:15.928611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff883a-e369-4b30-909b-ebad48e1ff70 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:16.024879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:16.058952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d40f-b5e7-43a2-ab0c-0e4e28308713 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.059256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d40f-b5e7-43a2-ab0c-0e4e28308713 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.059534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d40f-b5e7-43a2-ab0c-0e4e28308713 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.060908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 56/101] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:47:16.067580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2dcef-c298-4c6a-adf0-ca93d0edc2cf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.067874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2dcef-c298-4c6a-adf0-ca93d0edc2cf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.068161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2dcef-c298-4c6a-adf0-ca93d0edc2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.106981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3dd2dcef-c298-4c6a-adf0-ca93d0edc2cf None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:16.108269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3dd2dcef-c298-4c6a-adf0-ca93d0edc2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:16.137870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff883a-e369-4b30-909b-ebad48e1ff70 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:16.221043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 57/102] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:16.222439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb70fd2-cad0-4142-9e41-173015466be4 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.223267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb70fd2-cad0-4142-9e41-173015466be4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.223552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb70fd2-cad0-4142-9e41-173015466be4 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.224946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 58/103] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:16.227859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b6abf-9a3b-43cb-bf33-5f0475c29ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:16.228537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b6abf-9a3b-43cb-bf33-5f0475c29ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.228823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b6abf-9a3b-43cb-bf33-5f0475c29ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.229102 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b6abf-9a3b-43cb-bf33-5f0475c29ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.229859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b6abf-9a3b-43cb-bf33-5f0475c29ac2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:16.231039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82dfcc-a1f1-4f9d-be43-53d2eb52cc52 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.231321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82dfcc-a1f1-4f9d-be43-53d2eb52cc52 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.231595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82dfcc-a1f1-4f9d-be43-53d2eb52cc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.235235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b6abf-9a3b-43cb-bf33-5f0475c29ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:16.236628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b6abf-9a3b-43cb-bf33-5f0475c29ac2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:16.257872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-dc82dfcc-a1f1-4f9d-be43-53d2eb52cc52 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:16.259101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dc82dfcc-a1f1-4f9d-be43-53d2eb52cc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:16.260250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 46/104] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:16.261259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b6abf-9a3b-43cb-bf33-5f0475c29ac2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:16.333833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 59/105] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:16.334736 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82dfcc-a1f1-4f9d-be43-53d2eb52cc52 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:16.400359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:16.470391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82dfcc-a1f1-4f9d-be43-53d2eb52cc52 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:16.472576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b6abf-9a3b-43cb-bf33-5f0475c29ac2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:16.475617 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:16.509431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab0e23-ebd9-43c0-926e-731ca6238dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:16.509625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2ad63-44a2-4aea-a025-d0e93b127da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:16.510490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2ad63-44a2-4aea-a025-d0e93b127da0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.510889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2ad63-44a2-4aea-a025-d0e93b127da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.511177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2ad63-44a2-4aea-a025-d0e93b127da0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.511407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0e23-ebd9-43c0-926e-731ca6238dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.511875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0e23-ebd9-43c0-926e-731ca6238dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.512347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0e23-ebd9-43c0-926e-731ca6238dd3 None admin] PATH_INFO: `/v3/users/b71baf76a6154ab2b4134c4509189ddc` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.512507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2ad63-44a2-4aea-a025-d0e93b127da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:16.514148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2ad63-44a2-4aea-a025-d0e93b127da0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:16.516468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 47/106] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:16.516679 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2ad63-44a2-4aea-a025-d0e93b127da0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:16.529027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab0e23-ebd9-43c0-926e-731ca6238dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=b71baf76a6154ab2b4134c4509189ddc)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:16.530948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab0e23-ebd9-43c0-926e-731ca6238dd3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:16.532967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 60/107] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:16 2022] GET /identity/v3/users/b71baf76a6154ab2b4134c4509189ddc => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:16.534544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae5afe-235a-42d6-bc15-e1b7c8f7f065 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.535025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae5afe-235a-42d6-bc15-e1b7c8f7f065 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.535586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae5afe-235a-42d6-bc15-e1b7c8f7f065 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.537356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 61/108] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:16.538529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae5afe-235a-42d6-bc15-e1b7c8f7f065 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:16.627146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2ad63-44a2-4aea-a025-d0e93b127da0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:16.644380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:16.652880 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae5afe-235a-42d6-bc15-e1b7c8f7f065 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:16.664017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b763dbd-b08b-4d07-8b02-754abad1a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:16.664739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b763dbd-b08b-4d07-8b02-754abad1a205 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.665122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b763dbd-b08b-4d07-8b02-754abad1a205 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.665407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b763dbd-b08b-4d07-8b02-754abad1a205 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.678017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b763dbd-b08b-4d07-8b02-754abad1a205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-2b763dbd-b08b-4d07-8b02-754abad1a205 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:16.686342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:16.687183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:16.687936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 48/109] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:16.688555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b763dbd-b08b-4d07-8b02-754abad1a205 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:16.699433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bece5-9216-4c01-9c1e-41e44d7c0ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:16.700499 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bece5-9216-4c01-9c1e-41e44d7c0ce5 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.700973 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bece5-9216-4c01-9c1e-41e44d7c0ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.701440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bece5-9216-4c01-9c1e-41e44d7c0ce5 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.702419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74bece5-9216-4c01-9c1e-41e44d7c0ce5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:16.704337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74bece5-9216-4c01-9c1e-41e44d7c0ce5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:16.722090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:16.791828 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b763dbd-b08b-4d07-8b02-754abad1a205 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:16.801565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 62/110] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:47:16 2022] POST /identity/v3/projects => generated 302 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:16.804412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912b852-3563-4fe5-9c3b-f1a2024be6d3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.804937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912b852-3563-4fe5-9c3b-f1a2024be6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.805993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912b852-3563-4fe5-9c3b-f1a2024be6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.828448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f912b852-3563-4fe5-9c3b-f1a2024be6d3 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:16.830440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f912b852-3563-4fe5-9c3b-f1a2024be6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:16.856834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba44fc-7198-4b60-83fc-b7fc8cde0d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:16.857756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba44fc-7198-4b60-83fc-b7fc8cde0d46 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:16.858301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba44fc-7198-4b60-83fc-b7fc8cde0d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:16.858725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba44fc-7198-4b60-83fc-b7fc8cde0d46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:16.919297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 63/111] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:16.920301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912b852-3563-4fe5-9c3b-f1a2024be6d3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:17.007031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba44fc-7198-4b60-83fc-b7fc8cde0d46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:17.009348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba44fc-7198-4b60-83fc-b7fc8cde0d46 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:17.024690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:17.045854 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:17.057186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912b852-3563-4fe5-9c3b-f1a2024be6d3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:17.143059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85e217-c152-4d86-8a0f-5710c4aa872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:17.143797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85e217-c152-4d86-8a0f-5710c4aa872c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.144087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85e217-c152-4d86-8a0f-5710c4aa872c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.144455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85e217-c152-4d86-8a0f-5710c4aa872c None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.145443 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85e217-c152-4d86-8a0f-5710c4aa872c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:17.145759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 49/112] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:17.146106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85e217-c152-4d86-8a0f-5710c4aa872c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:17.147059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba44fc-7198-4b60-83fc-b7fc8cde0d46 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:17.149242 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85e217-c152-4d86-8a0f-5710c4aa872c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:17.157580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 64/113] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:17.158579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57036885-a058-412f-b232-82f92097f4ae None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.158876 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57036885-a058-412f-b232-82f92097f4ae None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.159159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57036885-a058-412f-b232-82f92097f4ae None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.160553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 65/114] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:17.164805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57036885-a058-412f-b232-82f92097f4ae None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:17.266276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba44fc-7198-4b60-83fc-b7fc8cde0d46 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:17.272082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57036885-a058-412f-b232-82f92097f4ae None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:17.298371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:17.304396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07f3c8-8ed6-40a6-998c-dc8a93e8d914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:17.305259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07f3c8-8ed6-40a6-998c-dc8a93e8d914 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.305611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07f3c8-8ed6-40a6-998c-dc8a93e8d914 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.305907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07f3c8-8ed6-40a6-998c-dc8a93e8d914 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.306262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedefbc-8e0c-45e0-9ee5-6bf45f63e3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:17.312103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedefbc-8e0c-45e0-9ee5-6bf45f63e3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.312103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedefbc-8e0c-45e0-9ee5-6bf45f63e3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.312103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedefbc-8e0c-45e0-9ee5-6bf45f63e3d7 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.312103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fedefbc-8e0c-45e0-9ee5-6bf45f63e3d7 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:17.312103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fedefbc-8e0c-45e0-9ee5-6bf45f63e3d7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:17.318140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 66/115] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:17 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:17.318140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedefbc-8e0c-45e0-9ee5-6bf45f63e3d7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:17.322743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07f3c8-8ed6-40a6-998c-dc8a93e8d914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:17.329302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7d07f3c8-8ed6-40a6-998c-dc8a93e8d914 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:17.330910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:17.331896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 50/116] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:17.338344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd46fb-5656-407a-8e9c-0c8133ce1de0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.339177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd46fb-5656-407a-8e9c-0c8133ce1de0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.339177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd46fb-5656-407a-8e9c-0c8133ce1de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.358479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-22fd46fb-5656-407a-8e9c-0c8133ce1de0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:17.359751 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-22fd46fb-5656-407a-8e9c-0c8133ce1de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:17.436448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedefbc-8e0c-45e0-9ee5-6bf45f63e3d7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:17.436751 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 51/117] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:17.438226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78df07-3e22-4c60-87bc-acd5390952f6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.438661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78df07-3e22-4c60-87bc-acd5390952f6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.439059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78df07-3e22-4c60-87bc-acd5390952f6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.440435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 52/118] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:17.441238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78df07-3e22-4c60-87bc-acd5390952f6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:17.483161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acbdfd-bc27-40bb-b874-8dd5c3ba7077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:17.484288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acbdfd-bc27-40bb-b874-8dd5c3ba7077 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.484713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acbdfd-bc27-40bb-b874-8dd5c3ba7077 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.485121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acbdfd-bc27-40bb-b874-8dd5c3ba7077 None admin] PATH_INFO: `/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.486489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acbdfd-bc27-40bb-b874-8dd5c3ba7077 None admin] RBAC: Authorizing `identity:get_user(user_id=26ad26ac4aea4a7b8fb923b47a154aa9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:17.488076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acbdfd-bc27-40bb-b874-8dd5c3ba7077 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:17.490343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 67/119] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:17 2022] GET /identity/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:17.491241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acbdfd-bc27-40bb-b874-8dd5c3ba7077 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:17.561098 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78df07-3e22-4c60-87bc-acd5390952f6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:17.575647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:17.585081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:17.598152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c5350-dafa-48be-b993-4bdfe30b8f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:17.598812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c5350-dafa-48be-b993-4bdfe30b8f6b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.599092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c5350-dafa-48be-b993-4bdfe30b8f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.599414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c5350-dafa-48be-b993-4bdfe30b8f6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.600241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c5350-dafa-48be-b993-4bdfe30b8f6b None admin] RBAC: Adding query filter params (role.id=43bec89a20b54bdeb265c47e7af8d7b6, scope.project.id=245d48325d884a83b0a57436df3e1418, user.id=b71baf76a6154ab2b4134c4509189ddc) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:17.600549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c5350-dafa-48be-b993-4bdfe30b8f6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:17.601603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acbdfd-bc27-40bb-b874-8dd5c3ba7077 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:17.602003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c5350-dafa-48be-b993-4bdfe30b8f6b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:17.614094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 53/120] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:17 2022] GET /identity/v3/role_assignments?user.id=b71baf76a6154ab2b4134c4509189ddc&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=43bec89a20b54bdeb265c47e7af8d7b6 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:17.614094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711503b-4a77-46fa-b4fa-70a41624e040 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.614094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711503b-4a77-46fa-b4fa-70a41624e040 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.614094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711503b-4a77-46fa-b4fa-70a41624e040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.634723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9711503b-4a77-46fa-b4fa-70a41624e040 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:17.635959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9711503b-4a77-46fa-b4fa-70a41624e040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:17.655136 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42841da4-3fb0-41a6-a06b-2235f358f6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:17.655822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42841da4-3fb0-41a6-a06b-2235f358f6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.656282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42841da4-3fb0-41a6-a06b-2235f358f6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.656383 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42841da4-3fb0-41a6-a06b-2235f358f6e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.657444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42841da4-3fb0-41a6-a06b-2235f358f6e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:17.657444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42841da4-3fb0-41a6-a06b-2235f358f6e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:17.658899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42841da4-3fb0-41a6-a06b-2235f358f6e5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:17.669232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 68/121] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:17.670218 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42841da4-3fb0-41a6-a06b-2235f358f6e5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:17.731845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 54/122] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:17.739762 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711503b-4a77-46fa-b4fa-70a41624e040 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:17.794605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42841da4-3fb0-41a6-a06b-2235f358f6e5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:17.832961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72373125-8ff9-4fea-ac3e-fc97860b477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:17.833699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72373125-8ff9-4fea-ac3e-fc97860b477c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.833983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72373125-8ff9-4fea-ac3e-fc97860b477c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.834313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72373125-8ff9-4fea-ac3e-fc97860b477c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.844127 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:17.848607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72373125-8ff9-4fea-ac3e-fc97860b477c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-72373125-8ff9-4fea-ac3e-fc97860b477c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:17.857244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:17.859156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:17.859935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 69/123] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:17.865199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72373125-8ff9-4fea-ac3e-fc97860b477c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:17.885958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711503b-4a77-46fa-b4fa-70a41624e040 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:17.904852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:17.925067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f275bf-dfe6-4582-854a-28c92318fc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:17.926377 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f275bf-dfe6-4582-854a-28c92318fc1a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:17.926918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f275bf-dfe6-4582-854a-28c92318fc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:17.927486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f275bf-dfe6-4582-854a-28c92318fc1a None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:17.928990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f275bf-dfe6-4582-854a-28c92318fc1a None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:17.930864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f275bf-dfe6-4582-854a-28c92318fc1a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:17.932763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 55/124] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:17 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:17.933929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f275bf-dfe6-4582-854a-28c92318fc1a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:17.976543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72373125-8ff9-4fea-ac3e-fc97860b477c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:18.012404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb46cf-f745-4f89-ae8c-7bfd39106cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:18.013330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb46cf-f745-4f89-ae8c-7bfd39106cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:18.013612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb46cf-f745-4f89-ae8c-7bfd39106cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:18.013882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb46cf-f745-4f89-ae8c-7bfd39106cf7 None admin] PATH_INFO: `/v3/users/654695f326044efdb73fe2956b7ce735` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:18.015124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb46cf-f745-4f89-ae8c-7bfd39106cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=654695f326044efdb73fe2956b7ce735)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:18.016592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb46cf-f745-4f89-ae8c-7bfd39106cf7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:18.018422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 70/125] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:17 2022] GET /identity/v3/users/654695f326044efdb73fe2956b7ce735 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:18.019305 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb46cf-f745-4f89-ae8c-7bfd39106cf7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:18.048276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f275bf-dfe6-4582-854a-28c92318fc1a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:18.087241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d60c63-de28-4008-a463-d90d08e73b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:18.088435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d60c63-de28-4008-a463-d90d08e73b35 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:18.088949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d60c63-de28-4008-a463-d90d08e73b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:18.089400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d60c63-de28-4008-a463-d90d08e73b35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:18.101820 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d60c63-de28-4008-a463-d90d08e73b35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:18.104169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-48d60c63-de28-4008-a463-d90d08e73b35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:18.111365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:18.116288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 56/126] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:18.117131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d60c63-de28-4008-a463-d90d08e73b35 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:18.147191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb46cf-f745-4f89-ae8c-7bfd39106cf7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:18.168979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:18.197863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61964000-a1ee-40ad-af56-18d66de27772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:18.198653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61964000-a1ee-40ad-af56-18d66de27772 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:18.199010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61964000-a1ee-40ad-af56-18d66de27772 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:18.199357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61964000-a1ee-40ad-af56-18d66de27772 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:18.200173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61964000-a1ee-40ad-af56-18d66de27772 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:18.200545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61964000-a1ee-40ad-af56-18d66de27772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:18.201994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61964000-a1ee-40ad-af56-18d66de27772 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:18.213815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 71/127] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:18.215638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61964000-a1ee-40ad-af56-18d66de27772 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:18.237070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d60c63-de28-4008-a463-d90d08e73b35 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:18.273958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4d4b2-d46e-45d1-b44f-21ccfb048394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:18.275161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4d4b2-d46e-45d1-b44f-21ccfb048394 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:18.275619 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4d4b2-d46e-45d1-b44f-21ccfb048394 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:18.276096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4d4b2-d46e-45d1-b44f-21ccfb048394 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418/users/26ad26ac4aea4a7b8fb923b47a154aa9/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:18.278363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4d4b2-d46e-45d1-b44f-21ccfb048394 None admin] RBAC: Authorizing `identity:create_grant(project_id=245d48325d884a83b0a57436df3e1418, user_id=26ad26ac4aea4a7b8fb923b47a154aa9, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:18.279997 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4d4b2-d46e-45d1-b44f-21ccfb048394 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:18.342143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61964000-a1ee-40ad-af56-18d66de27772 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:18.353448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 57/128] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:47:18 2022] PUT /identity/v3/projects/245d48325d884a83b0a57436df3e1418/users/26ad26ac4aea4a7b8fb923b47a154aa9/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:18.356385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4d4b2-d46e-45d1-b44f-21ccfb048394 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:18.372137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:18.381982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a44f8e-1fb0-4d0f-bdf7-8b45ddd4ca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:18.382918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a44f8e-1fb0-4d0f-bdf7-8b45ddd4ca40 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:18.383203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a44f8e-1fb0-4d0f-bdf7-8b45ddd4ca40 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:18.383482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a44f8e-1fb0-4d0f-bdf7-8b45ddd4ca40 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:18.384811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a44f8e-1fb0-4d0f-bdf7-8b45ddd4ca40 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:18.387519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a44f8e-1fb0-4d0f-bdf7-8b45ddd4ca40 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:18.389523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 72/129] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:18 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:18.390638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a44f8e-1fb0-4d0f-bdf7-8b45ddd4ca40 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:18.514473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a44f8e-1fb0-4d0f-bdf7-8b45ddd4ca40 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:18.554734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e25dc-dec7-46c5-9f38-7d52e856b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:18.555748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e25dc-dec7-46c5-9f38-7d52e856b8bf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:18.556035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e25dc-dec7-46c5-9f38-7d52e856b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:18.556312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e25dc-dec7-46c5-9f38-7d52e856b8bf None admin] PATH_INFO: `/v3/users/654695f326044efdb73fe2956b7ce735` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:18.557549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e25dc-dec7-46c5-9f38-7d52e856b8bf None admin] RBAC: Authorizing `identity:get_user(user_id=654695f326044efdb73fe2956b7ce735)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:18.559073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e25dc-dec7-46c5-9f38-7d52e856b8bf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:18.561114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 73/130] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:18 2022] GET /identity/v3/users/654695f326044efdb73fe2956b7ce735 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:18.567246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e25dc-dec7-46c5-9f38-7d52e856b8bf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:18.640021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4d4b2-d46e-45d1-b44f-21ccfb048394 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:18.644988 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:18.666421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:18.687554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441fc08-ce76-4af2-b274-c4424cda5120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:18.688757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441fc08-ce76-4af2-b274-c4424cda5120 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:18.689186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441fc08-ce76-4af2-b274-c4424cda5120 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:18.689734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441fc08-ce76-4af2-b274-c4424cda5120 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:18.691229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441fc08-ce76-4af2-b274-c4424cda5120 None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:18.692237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441fc08-ce76-4af2-b274-c4424cda5120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:18.693794 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441fc08-ce76-4af2-b274-c4424cda5120 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:18.694227 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e25dc-dec7-46c5-9f38-7d52e856b8bf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:18.704278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 58/131] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:18.705799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441fc08-ce76-4af2-b274-c4424cda5120 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:18.747283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d6985-f85e-4c1d-a5ea-4dba39978e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:18.748247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d6985-f85e-4c1d-a5ea-4dba39978e57 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:18.748536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d6985-f85e-4c1d-a5ea-4dba39978e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:18.748815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d6985-f85e-4c1d-a5ea-4dba39978e57 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418/users/654695f326044efdb73fe2956b7ce735/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:18.752455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d6985-f85e-4c1d-a5ea-4dba39978e57 None admin] RBAC: Authorizing `identity:create_grant(project_id=245d48325d884a83b0a57436df3e1418, user_id=654695f326044efdb73fe2956b7ce735, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:18.754529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d6985-f85e-4c1d-a5ea-4dba39978e57 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:18.842365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441fc08-ce76-4af2-b274-c4424cda5120 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:18.878146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 74/132] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:47:18 2022] PUT /identity/v3/projects/245d48325d884a83b0a57436df3e1418/users/654695f326044efdb73fe2956b7ce735/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:18.880120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d6985-f85e-4c1d-a5ea-4dba39978e57 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:18.900911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79938100-8a41-4ef9-9909-ce91b14afa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:18.901698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79938100-8a41-4ef9-9909-ce91b14afa61 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:18.901976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79938100-8a41-4ef9-9909-ce91b14afa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:18.902274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79938100-8a41-4ef9-9909-ce91b14afa61 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:18.904773 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79938100-8a41-4ef9-9909-ce91b14afa61 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:18.907072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79938100-8a41-4ef9-9909-ce91b14afa61 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:18.911647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 59/133] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:18 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:18.917740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79938100-8a41-4ef9-9909-ce91b14afa61 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:18.978642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:19.114679 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79938100-8a41-4ef9-9909-ce91b14afa61 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:19.140833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d6985-f85e-4c1d-a5ea-4dba39978e57 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:19.168135 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:19.174727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec5ce9-6d5c-4189-a6e5-1a4aac3f2f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:19.175758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec5ce9-6d5c-4189-a6e5-1a4aac3f2f2c None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:19.176707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec5ce9-6d5c-4189-a6e5-1a4aac3f2f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:19.177051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec5ce9-6d5c-4189-a6e5-1a4aac3f2f2c None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418/users/654695f326044efdb73fe2956b7ce735/roles/120eeffad7094e10a69166d19aaa6439` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:19.180224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec5ce9-6d5c-4189-a6e5-1a4aac3f2f2c None admin] RBAC: Authorizing `identity:create_grant(project_id=245d48325d884a83b0a57436df3e1418, user_id=654695f326044efdb73fe2956b7ce735, role_id=120eeffad7094e10a69166d19aaa6439)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:19.184752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c72f6-6cd3-4caf-b0e1-2a7ff5823ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:19.184868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec5ce9-6d5c-4189-a6e5-1a4aac3f2f2c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:19.184940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c72f6-6cd3-4caf-b0e1-2a7ff5823ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:19.184940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c72f6-6cd3-4caf-b0e1-2a7ff5823ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:19.184940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c72f6-6cd3-4caf-b0e1-2a7ff5823ed6 None admin] PATH_INFO: `/v3/users/654695f326044efdb73fe2956b7ce735` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:19.184940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c72f6-6cd3-4caf-b0e1-2a7ff5823ed6 None admin] RBAC: Authorizing `identity:get_user(user_id=654695f326044efdb73fe2956b7ce735)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:19.185592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c72f6-6cd3-4caf-b0e1-2a7ff5823ed6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:19.187724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 75/134] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:18 2022] GET /identity/v3/users/654695f326044efdb73fe2956b7ce735 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:19.191185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0248d-0658-4b1a-aa76-85c2c0e25aa4 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:19.191504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0248d-0658-4b1a-aa76-85c2c0e25aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:19.191908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0248d-0658-4b1a-aa76-85c2c0e25aa4 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:19.193339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 76/135] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:19.194495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0248d-0658-4b1a-aa76-85c2c0e25aa4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:19.305008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0248d-0658-4b1a-aa76-85c2c0e25aa4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:19.340409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 60/136] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:47:18 2022] PUT /identity/v3/projects/245d48325d884a83b0a57436df3e1418/users/654695f326044efdb73fe2956b7ce735/roles/120eeffad7094e10a69166d19aaa6439 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 08:47:19.341790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2cc07-c70a-47de-aa6f-c80717caccec None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:19.342099 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2cc07-c70a-47de-aa6f-c80717caccec None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:19.342380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2cc07-c70a-47de-aa6f-c80717caccec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:19.352217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c509f62-3903-4534-8c54-ec8ca3fe2b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:19.353101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c509f62-3903-4534-8c54-ec8ca3fe2b70 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:19.353386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c509f62-3903-4534-8c54-ec8ca3fe2b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:19.353672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c509f62-3903-4534-8c54-ec8ca3fe2b70 None admin] PATH_INFO: `/v3/projects/94ce177c584046218878ea073342cd16` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:19.354999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c509f62-3903-4534-8c54-ec8ca3fe2b70 None admin] RBAC: Authorizing `identity:get_project(project_id=94ce177c584046218878ea073342cd16)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:19.356761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c509f62-3903-4534-8c54-ec8ca3fe2b70 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:19.358599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 77/137] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:19 2022] GET /identity/v3/projects/94ce177c584046218878ea073342cd16 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:19.365296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c509f62-3903-4534-8c54-ec8ca3fe2b70 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:19.394297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b7a2cc07-c70a-47de-aa6f-c80717caccec None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:19.395302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b7a2cc07-c70a-47de-aa6f-c80717caccec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:19.499407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:19.563584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 61/138] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:19.588207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c509f62-3903-4534-8c54-ec8ca3fe2b70 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:19.605343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2cc07-c70a-47de-aa6f-c80717caccec None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:19.639693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:19.653114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6a38f-318e-4b18-b90b-822dd2e80b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:19.653114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6a38f-318e-4b18-b90b-822dd2e80b17 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:19.653114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6a38f-318e-4b18-b90b-822dd2e80b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:19.653423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6a38f-318e-4b18-b90b-822dd2e80b17 None admin] PATH_INFO: `/v3/projects/94ce177c584046218878ea073342cd16/users/654695f326044efdb73fe2956b7ce735/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:19.655436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6a38f-318e-4b18-b90b-822dd2e80b17 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce177c584046218878ea073342cd16, user_id=654695f326044efdb73fe2956b7ce735, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:19.656915 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6a38f-318e-4b18-b90b-822dd2e80b17 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:19.752405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2cc07-c70a-47de-aa6f-c80717caccec None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:19.757116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 78/139] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:47:19 2022] PUT /identity/v3/projects/94ce177c584046218878ea073342cd16/users/654695f326044efdb73fe2956b7ce735/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:19.808111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58c88e-0cbc-4739-b7c3-5506fa88f324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:19.809231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58c88e-0cbc-4739-b7c3-5506fa88f324 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:19.809691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58c88e-0cbc-4739-b7c3-5506fa88f324 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:19.810227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58c88e-0cbc-4739-b7c3-5506fa88f324 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:19.812668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58c88e-0cbc-4739-b7c3-5506fa88f324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:19.814396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58c88e-0cbc-4739-b7c3-5506fa88f324 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:19.817178 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 62/140] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:19.822179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6a38f-318e-4b18-b90b-822dd2e80b17 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:19.848477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d2424-7d5e-437d-a537-1ce750cace2b None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:19.849094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d2424-7d5e-437d-a537-1ce750cace2b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:19.849548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d2424-7d5e-437d-a537-1ce750cace2b None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:19.851471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 63/141] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:47:19.856559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad7ea2-01d8-4a96-935f-09e1214d7f52 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:19.857087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad7ea2-01d8-4a96-935f-09e1214d7f52 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:19.857567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad7ea2-01d8-4a96-935f-09e1214d7f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:19.889141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f7ad7ea2-01d8-4a96-935f-09e1214d7f52 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:19.890732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f7ad7ea2-01d8-4a96-935f-09e1214d7f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:20.031389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 64/142] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:20.031646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6a38f-318e-4b18-b90b-822dd2e80b17 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:20.039549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:20.051242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad7ea2-01d8-4a96-935f-09e1214d7f52 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:20.074026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc326606-82ef-45bd-a5d4-6d5774e8f2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:20.074824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326606-82ef-45bd-a5d4-6d5774e8f2b7 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:20.075112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326606-82ef-45bd-a5d4-6d5774e8f2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:20.075388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326606-82ef-45bd-a5d4-6d5774e8f2b7 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:20.076132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc326606-82ef-45bd-a5d4-6d5774e8f2b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:20.077937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc326606-82ef-45bd-a5d4-6d5774e8f2b7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:20.093820 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:20.179197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad7ea2-01d8-4a96-935f-09e1214d7f52 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:20.198310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 79/143] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:47:19 2022] POST /identity/v3/users => generated 283 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:20.235805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56e401-d763-41b4-88e7-adc575f0a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:20.236717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56e401-d763-41b4-88e7-adc575f0a720 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:20.237001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56e401-d763-41b4-88e7-adc575f0a720 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:20.237279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56e401-d763-41b4-88e7-adc575f0a720 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:20.253439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56e401-d763-41b4-88e7-adc575f0a720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-4f56e401-d763-41b4-88e7-adc575f0a720 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:20.261901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:20.265202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:20.265995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 65/144] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:20.271249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc326606-82ef-45bd-a5d4-6d5774e8f2b7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:20.406788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc326606-82ef-45bd-a5d4-6d5774e8f2b7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:20.451789 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bb83a-fb92-4bc5-ba7d-e60851787192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:20.452468 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bb83a-fb92-4bc5-ba7d-e60851787192 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:20.452752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bb83a-fb92-4bc5-ba7d-e60851787192 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:20.453029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bb83a-fb92-4bc5-ba7d-e60851787192 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:20.453782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bb83a-fb92-4bc5-ba7d-e60851787192 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:20.454113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bb83a-fb92-4bc5-ba7d-e60851787192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:20.455584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bb83a-fb92-4bc5-ba7d-e60851787192 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:20.470158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 80/145] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:20.475824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56e401-d763-41b4-88e7-adc575f0a720 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:20.574086 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:20.599838 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56e401-d763-41b4-88e7-adc575f0a720 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:20.643466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a8f37-898f-4fb3-adc1-24fb47c109bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:20.644431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a8f37-898f-4fb3-adc1-24fb47c109bf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:20.645263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a8f37-898f-4fb3-adc1-24fb47c109bf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:20.645545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a8f37-898f-4fb3-adc1-24fb47c109bf None admin] PATH_INFO: `/v3/users/b71baf76a6154ab2b4134c4509189ddc` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:20.646787 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a8f37-898f-4fb3-adc1-24fb47c109bf None admin] RBAC: Authorizing `identity:get_user(user_id=b71baf76a6154ab2b4134c4509189ddc)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:20.648249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a8f37-898f-4fb3-adc1-24fb47c109bf None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:20.650215 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 66/146] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:20 2022] GET /identity/v3/users/b71baf76a6154ab2b4134c4509189ddc => generated 289 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:20.655624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bb83a-fb92-4bc5-ba7d-e60851787192 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:20.699810 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:20.784626 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bb83a-fb92-4bc5-ba7d-e60851787192 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:20.831325 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3ec35-e13b-46f5-8b4a-e94e18aec810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:20.832408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3ec35-e13b-46f5-8b4a-e94e18aec810 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:20.833060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3ec35-e13b-46f5-8b4a-e94e18aec810 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:20.833338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3ec35-e13b-46f5-8b4a-e94e18aec810 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:20.834571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3ec35-e13b-46f5-8b4a-e94e18aec810 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:20.836021 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3ec35-e13b-46f5-8b4a-e94e18aec810 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:20.838717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 81/147] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:20 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:20.843298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a8f37-898f-4fb3-adc1-24fb47c109bf None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:20.986409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a8f37-898f-4fb3-adc1-24fb47c109bf None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:21.030985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82d6a4-18a3-44c7-9b63-0955645bdf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:21.031875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82d6a4-18a3-44c7-9b63-0955645bdf15 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:21.032161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82d6a4-18a3-44c7-9b63-0955645bdf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:21.032433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82d6a4-18a3-44c7-9b63-0955645bdf15 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418/users/b71baf76a6154ab2b4134c4509189ddc/roles/43bec89a20b54bdeb265c47e7af8d7b6` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:21.037640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82d6a4-18a3-44c7-9b63-0955645bdf15 None admin] RBAC: Authorizing `identity:create_grant(project_id=245d48325d884a83b0a57436df3e1418, user_id=b71baf76a6154ab2b4134c4509189ddc, role_id=43bec89a20b54bdeb265c47e7af8d7b6)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:21.045674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82d6a4-18a3-44c7-9b63-0955645bdf15 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:21.153457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 67/148] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:47:20 2022] PUT /identity/v3/projects/245d48325d884a83b0a57436df3e1418/users/b71baf76a6154ab2b4134c4509189ddc/roles/43bec89a20b54bdeb265c47e7af8d7b6 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:21.469536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df95af-ea7b-4a07-a4aa-141ef1aee6ec None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:21.470163 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df95af-ea7b-4a07-a4aa-141ef1aee6ec None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:21.470643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df95af-ea7b-4a07-a4aa-141ef1aee6ec None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:21.472357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 82/149] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:21.478243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dc48b-ee8b-4aaf-b02a-407b32d2f691 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:21.478526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dc48b-ee8b-4aaf-b02a-407b32d2f691 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:21.478811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dc48b-ee8b-4aaf-b02a-407b32d2f691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:21.501171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bf3dc48b-ee8b-4aaf-b02a-407b32d2f691 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:21.502417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bf3dc48b-ee8b-4aaf-b02a-407b32d2f691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:21.506463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:21.582171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 68/150] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:21.619836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df95af-ea7b-4a07-a4aa-141ef1aee6ec None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:21.762064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df95af-ea7b-4a07-a4aa-141ef1aee6ec None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:21.769756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:21.799583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf8019-b52f-49a7-94e8-8f239187cbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:21.800380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf8019-b52f-49a7-94e8-8f239187cbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:21.801267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf8019-b52f-49a7-94e8-8f239187cbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:21.801546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf8019-b52f-49a7-94e8-8f239187cbe2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:21.813608 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf8019-b52f-49a7-94e8-8f239187cbe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-cddf8019-b52f-49a7-94e8-8f239187cbe2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:21.822644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:21.823522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:21.824741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 83/151] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:21.829641 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dc48b-ee8b-4aaf-b02a-407b32d2f691 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:21.844759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb950a3f-356c-4e37-a9a4-717bd690428f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:21.845059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb950a3f-356c-4e37-a9a4-717bd690428f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:21.845314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb950a3f-356c-4e37-a9a4-717bd690428f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:21.848691 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 84/152] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:21.854825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8df9a-ec36-4823-a9fd-d8173936d137 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:21.855106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8df9a-ec36-4823-a9fd-d8173936d137 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:21.855390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8df9a-ec36-4823-a9fd-d8173936d137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:21.889559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-08d8df9a-ec36-4823-a9fd-d8173936d137 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:21.890784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-08d8df9a-ec36-4823-a9fd-d8173936d137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:21.962081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dc48b-ee8b-4aaf-b02a-407b32d2f691 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:22.012299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c21a6-58cb-43d1-bd63-5a95043b9782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:22.012994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c21a6-58cb-43d1-bd63-5a95043b9782 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:22.013820 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c21a6-58cb-43d1-bd63-5a95043b9782 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:22.014126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c21a6-58cb-43d1-bd63-5a95043b9782 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:22.014895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c21a6-58cb-43d1-bd63-5a95043b9782 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:22.015207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c21a6-58cb-43d1-bd63-5a95043b9782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:22.016603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c21a6-58cb-43d1-bd63-5a95043b9782 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:22.018439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 85/153] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:22.028471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 69/154] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:22.034009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8df9a-ec36-4823-a9fd-d8173936d137 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:22.056468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c21a6-58cb-43d1-bd63-5a95043b9782 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:22.059688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:22.217100 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:22.228481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8df9a-ec36-4823-a9fd-d8173936d137 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:22.267764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c21a6-58cb-43d1-bd63-5a95043b9782 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:22.294759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781d61a-a49d-43c4-b255-2d5640a66c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:22.295987 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781d61a-a49d-43c4-b255-2d5640a66c28 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:22.296737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781d61a-a49d-43c4-b255-2d5640a66c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:22.297207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781d61a-a49d-43c4-b255-2d5640a66c28 None admin] PATH_INFO: `/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:22.300503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781d61a-a49d-43c4-b255-2d5640a66c28 None admin] RBAC: Authorizing `identity:get_user(user_id=26ad26ac4aea4a7b8fb923b47a154aa9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:22.302952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781d61a-a49d-43c4-b255-2d5640a66c28 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:22.307643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 86/155] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:22 2022] GET /identity/v3/users/26ad26ac4aea4a7b8fb923b47a154aa9 => generated 247 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:22.312007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781d61a-a49d-43c4-b255-2d5640a66c28 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:22.378845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3ffce-a879-4905-b976-ed33da788b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:22.379763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3ffce-a879-4905-b976-ed33da788b29 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:22.380042 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3ffce-a879-4905-b976-ed33da788b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:22.380320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3ffce-a879-4905-b976-ed33da788b29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:22.415010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3ffce-a879-4905-b976-ed33da788b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b5d3ffce-a879-4905-b976-ed33da788b29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:22.431220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:22.432121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:22.433403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 70/156] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:22.438250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3ffce-a879-4905-b976-ed33da788b29 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:22.447480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781d61a-a49d-43c4-b255-2d5640a66c28 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:22.490403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae490c-208c-461c-af0b-28e7853aced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:22.491401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae490c-208c-461c-af0b-28e7853aced9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:22.491884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae490c-208c-461c-af0b-28e7853aced9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:22.492356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae490c-208c-461c-af0b-28e7853aced9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:22.537791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:22.581612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:22.604134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae490c-208c-461c-af0b-28e7853aced9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:22.605964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae490c-208c-461c-af0b-28e7853aced9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:22.651417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3ffce-a879-4905-b976-ed33da788b29 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:22.708832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c5df5-21fc-4aa1-a9a3-f7fb0c3226d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:22.709524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5df5-21fc-4aa1-a9a3-f7fb0c3226d5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:22.710003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5df5-21fc-4aa1-a9a3-f7fb0c3226d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:22.710458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5df5-21fc-4aa1-a9a3-f7fb0c3226d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:22.711420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c5df5-21fc-4aa1-a9a3-f7fb0c3226d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:22.711882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c5df5-21fc-4aa1-a9a3-f7fb0c3226d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:22.713421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c5df5-21fc-4aa1-a9a3-f7fb0c3226d5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:22.728385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 71/157] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:22.733703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c5df5-21fc-4aa1-a9a3-f7fb0c3226d5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:22.850817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 87/158] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 08:47:22.856133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae490c-208c-461c-af0b-28e7853aced9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:22.881601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c5df5-21fc-4aa1-a9a3-f7fb0c3226d5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:22.915320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:22.926540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f01a8-d730-4e9a-bb84-9d51fc287917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:22.929449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f01a8-d730-4e9a-bb84-9d51fc287917 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:22.929449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f01a8-d730-4e9a-bb84-9d51fc287917 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:22.929449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f01a8-d730-4e9a-bb84-9d51fc287917 None admin] PATH_INFO: `/v3/users/654695f326044efdb73fe2956b7ce735` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:22.929657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f01a8-d730-4e9a-bb84-9d51fc287917 None admin] RBAC: Authorizing `identity:get_user(user_id=654695f326044efdb73fe2956b7ce735)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:22.935462 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f01a8-d730-4e9a-bb84-9d51fc287917 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:22.937296 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 72/159] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:22 2022] GET /identity/v3/users/654695f326044efdb73fe2956b7ce735 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:22.943953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f01a8-d730-4e9a-bb84-9d51fc287917 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:23.039038 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:23.113878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae490c-208c-461c-af0b-28e7853aced9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:23.165607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f01a8-d730-4e9a-bb84-9d51fc287917 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:23.202834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe35cca-49b6-4467-8692-5ae03f8bf7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:23.203975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe35cca-49b6-4467-8692-5ae03f8bf7ac None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:23.204395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe35cca-49b6-4467-8692-5ae03f8bf7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:23.204814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe35cca-49b6-4467-8692-5ae03f8bf7ac None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:23.206299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe35cca-49b6-4467-8692-5ae03f8bf7ac None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:23.208482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe35cca-49b6-4467-8692-5ae03f8bf7ac None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:23.210367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 88/160] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:22 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:23.214142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f3604-3208-424d-bedc-555cb7c62744 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:23.214585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f3604-3208-424d-bedc-555cb7c62744 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:23.215019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f3604-3208-424d-bedc-555cb7c62744 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:23.219035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 89/161] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:23.219971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f3604-3208-424d-bedc-555cb7c62744 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:23.247402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3036a-f633-4bb7-ab69-1f066303899f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:23.248073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3036a-f633-4bb7-ab69-1f066303899f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:23.248359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3036a-f633-4bb7-ab69-1f066303899f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:23.248637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3036a-f633-4bb7-ab69-1f066303899f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:23.355319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f3604-3208-424d-bedc-555cb7c62744 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:23.395119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:23.402481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:23.408027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3036a-f633-4bb7-ab69-1f066303899f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:23.409910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3036a-f633-4bb7-ab69-1f066303899f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:23.410753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30dcd3-4d4c-428c-a544-b667a7c366c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:23.411193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30dcd3-4d4c-428c-a544-b667a7c366c1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:23.411558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30dcd3-4d4c-428c-a544-b667a7c366c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:23.411832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30dcd3-4d4c-428c-a544-b667a7c366c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:23.412685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30dcd3-4d4c-428c-a544-b667a7c366c1 None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=245d48325d884a83b0a57436df3e1418, user.id=26ad26ac4aea4a7b8fb923b47a154aa9) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:23.412994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30dcd3-4d4c-428c-a544-b667a7c366c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:23.414543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30dcd3-4d4c-428c-a544-b667a7c366c1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:23.426536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 90/162] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:23 2022] GET /identity/v3/role_assignments?user.id=26ad26ac4aea4a7b8fb923b47a154aa9&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:23.427959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe5f11-2ed0-483c-aec3-08079d37ed70 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:23.428548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe5f11-2ed0-483c-aec3-08079d37ed70 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:23.428548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe5f11-2ed0-483c-aec3-08079d37ed70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:23.459210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bdbe5f11-2ed0-483c-aec3-08079d37ed70 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:23.460498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bdbe5f11-2ed0-483c-aec3-08079d37ed70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:23.572579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 91/163] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:23.573969 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e757b-ed0c-4adc-a254-02e7a105ce05 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:23.574298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e757b-ed0c-4adc-a254-02e7a105ce05 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:23.574578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e757b-ed0c-4adc-a254-02e7a105ce05 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:23.576050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 92/164] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:23.585764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6b39-af78-485a-9fed-db2c90866f67 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:23.586089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6b39-af78-485a-9fed-db2c90866f67 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:23.586347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6b39-af78-485a-9fed-db2c90866f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:23.627359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5e9e6b39-af78-485a-9fed-db2c90866f67 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:23.628564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5e9e6b39-af78-485a-9fed-db2c90866f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:23.628904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 73/165] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 08:47:23.655209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3036a-f633-4bb7-ab69-1f066303899f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:23.777853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 93/166] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 13 08:47:23.782817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6b39-af78-485a-9fed-db2c90866f67 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:23.808011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:23.817484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3036a-f633-4bb7-ab69-1f066303899f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:23.857845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b83f4-f188-4bef-8cf0-8bac1bf6762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:23.859231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b83f4-f188-4bef-8cf0-8bac1bf6762b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:23.859616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b83f4-f188-4bef-8cf0-8bac1bf6762b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:23.859998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b83f4-f188-4bef-8cf0-8bac1bf6762b None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:23.861313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b83f4-f188-4bef-8cf0-8bac1bf6762b None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:23.863705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b83f4-f188-4bef-8cf0-8bac1bf6762b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:23.865436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 74/167] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:23 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:23.866454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b83f4-f188-4bef-8cf0-8bac1bf6762b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:23.939452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:23.980098 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b83f4-f188-4bef-8cf0-8bac1bf6762b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:24.016803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f06586-d320-4728-9f60-ca7021d869e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:24.017718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f06586-d320-4728-9f60-ca7021d869e2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:24.018012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f06586-d320-4728-9f60-ca7021d869e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:24.018418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f06586-d320-4728-9f60-ca7021d869e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:24.031525 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f06586-d320-4728-9f60-ca7021d869e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:24.032387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6b39-af78-485a-9fed-db2c90866f67 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-c4f06586-d320-4728-9f60-ca7021d869e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:24.040397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:24.041456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:24.042289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 75/168] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:24.042955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f06586-d320-4728-9f60-ca7021d869e2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:24.073460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:24.085598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0cfd3-1ce5-499f-9de0-df084fd18c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:24.090707 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0cfd3-1ce5-499f-9de0-df084fd18c9b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:24.091003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0cfd3-1ce5-499f-9de0-df084fd18c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:24.091291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0cfd3-1ce5-499f-9de0-df084fd18c9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:24.105961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0cfd3-1ce5-499f-9de0-df084fd18c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-11d0cfd3-1ce5-499f-9de0-df084fd18c9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:24.119728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:24.120712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 94/169] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:24.122216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0cfd3-1ce5-499f-9de0-df084fd18c9b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:24.157360 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f06586-d320-4728-9f60-ca7021d869e2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:24.215461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8297c-69bd-45c6-b993-9835fda06714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:24.216052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8297c-69bd-45c6-b993-9835fda06714 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:24.216335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8297c-69bd-45c6-b993-9835fda06714 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:24.216604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8297c-69bd-45c6-b993-9835fda06714 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:24.217556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8297c-69bd-45c6-b993-9835fda06714 None admin] RBAC: Adding query filter params (role.id=a70b63ede7014fa9bbcb4e03275a2664, scope.project.id=245d48325d884a83b0a57436df3e1418, user.id=654695f326044efdb73fe2956b7ce735) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:24.217979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8297c-69bd-45c6-b993-9835fda06714 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:24.219469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8297c-69bd-45c6-b993-9835fda06714 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:24.230715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 76/170] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:24 2022] GET /identity/v3/role_assignments?user.id=654695f326044efdb73fe2956b7ce735&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:24.231577 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8297c-69bd-45c6-b993-9835fda06714 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:24.275860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0cfd3-1ce5-499f-9de0-df084fd18c9b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:24.305066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:24.329502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc02ec9-9c53-4af8-8219-bb71d59efe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:24.330221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc02ec9-9c53-4af8-8219-bb71d59efe18 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:24.330515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc02ec9-9c53-4af8-8219-bb71d59efe18 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:24.331060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc02ec9-9c53-4af8-8219-bb71d59efe18 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:24.331597 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc02ec9-9c53-4af8-8219-bb71d59efe18 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:24.332726 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc02ec9-9c53-4af8-8219-bb71d59efe18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:24.334263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc02ec9-9c53-4af8-8219-bb71d59efe18 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:24.345670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 95/171] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:24.351265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc02ec9-9c53-4af8-8219-bb71d59efe18 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:24.374505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:24.380984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8297c-69bd-45c6-b993-9835fda06714 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:24.429666 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d4c09-e247-4fa3-834f-c98c5935e385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:24.430420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d4c09-e247-4fa3-834f-c98c5935e385 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:24.430703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d4c09-e247-4fa3-834f-c98c5935e385 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:24.430977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d4c09-e247-4fa3-834f-c98c5935e385 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:24.431825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d4c09-e247-4fa3-834f-c98c5935e385 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:24.432134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d4c09-e247-4fa3-834f-c98c5935e385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:24.433716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d4c09-e247-4fa3-834f-c98c5935e385 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:24.445408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 77/172] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:24.451247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d4c09-e247-4fa3-834f-c98c5935e385 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:24.610171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc02ec9-9c53-4af8-8219-bb71d59efe18 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:24.649146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6eae9-012d-4159-af55-4d432068ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:24.650183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6eae9-012d-4159-af55-4d432068ac7c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:24.650468 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6eae9-012d-4159-af55-4d432068ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:24.650743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6eae9-012d-4159-af55-4d432068ac7c None admin] PATH_INFO: `/v3/users/654695f326044efdb73fe2956b7ce735` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:24.652203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6eae9-012d-4159-af55-4d432068ac7c None admin] RBAC: Authorizing `identity:get_user(user_id=654695f326044efdb73fe2956b7ce735)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:24.654195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6eae9-012d-4159-af55-4d432068ac7c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:24.655981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 96/173] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:24 2022] GET /identity/v3/users/654695f326044efdb73fe2956b7ce735 => generated 275 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:24.657158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dbbe7-5c05-4f4a-8415-a7127a0df9bd None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:24.657440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dbbe7-5c05-4f4a-8415-a7127a0df9bd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:24.657716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dbbe7-5c05-4f4a-8415-a7127a0df9bd None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:24.658774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d4c09-e247-4fa3-834f-c98c5935e385 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:24.659242 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 97/174] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:24.661621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dbbe7-5c05-4f4a-8415-a7127a0df9bd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:24.712016 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50e4f5-b100-44a4-9c9c-1d46dfb87309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:24.713152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e4f5-b100-44a4-9c9c-1d46dfb87309 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:24.713571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e4f5-b100-44a4-9c9c-1d46dfb87309 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:24.713996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e4f5-b100-44a4-9c9c-1d46dfb87309 None admin] PATH_INFO: `/v3/users/654695f326044efdb73fe2956b7ce735` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:24.718819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:24.719579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50e4f5-b100-44a4-9c9c-1d46dfb87309 None admin] RBAC: Authorizing `identity:get_user(user_id=654695f326044efdb73fe2956b7ce735)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:24.721310 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50e4f5-b100-44a4-9c9c-1d46dfb87309 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:24.723836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 78/175] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:24 2022] GET /identity/v3/users/654695f326044efdb73fe2956b7ce735 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:24.725930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e2260-f3a6-4c96-a11d-69edaefe3842 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:24.726446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e2260-f3a6-4c96-a11d-69edaefe3842 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:24.726862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e2260-f3a6-4c96-a11d-69edaefe3842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:24.757506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-056e2260-f3a6-4c96-a11d-69edaefe3842 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:24.758929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-056e2260-f3a6-4c96-a11d-69edaefe3842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:24.775917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:24.776212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dbbe7-5c05-4f4a-8415-a7127a0df9bd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:24.813303 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c6864-8342-4094-bcc1-e2deed00d04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:24.814160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c6864-8342-4094-bcc1-e2deed00d04a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:24.814542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c6864-8342-4094-bcc1-e2deed00d04a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:24.815051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c6864-8342-4094-bcc1-e2deed00d04a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:24.869253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 79/176] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:24.870492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e2260-f3a6-4c96-a11d-69edaefe3842 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:24.898073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c6864-8342-4094-bcc1-e2deed00d04a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:24.898073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c6864-8342-4094-bcc1-e2deed00d04a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:24.983511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:25.009662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 98/177] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:25.009662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c6864-8342-4094-bcc1-e2deed00d04a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:25.037558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e2260-f3a6-4c96-a11d-69edaefe3842 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:25.076316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef30f84-cd3f-43db-9bbb-47062b2b7732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:25.077540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30f84-cd3f-43db-9bbb-47062b2b7732 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.077872 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30f84-cd3f-43db-9bbb-47062b2b7732 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.078447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30f84-cd3f-43db-9bbb-47062b2b7732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.097513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:25.113015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c6864-8342-4094-bcc1-e2deed00d04a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:25.150472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b7e16-835b-414e-8edb-f479ebc2a618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:25.151191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b7e16-835b-414e-8edb-f479ebc2a618 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.151477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b7e16-835b-414e-8edb-f479ebc2a618 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.151751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b7e16-835b-414e-8edb-f479ebc2a618 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.163269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b7e16-835b-414e-8edb-f479ebc2a618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:25.169195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef30f84-cd3f-43db-9bbb-47062b2b7732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:25.170822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef30f84-cd3f-43db-9bbb-47062b2b7732 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d17b7e16-835b-414e-8edb-f479ebc2a618 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:25.171288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:25.172425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:25.173231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:25.178328 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 99/178] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:25.179231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b7e16-835b-414e-8edb-f479ebc2a618 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:25.189223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:25.290297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b7e16-835b-414e-8edb-f479ebc2a618 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:25.310064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 80/179] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:25.310064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e24c8-14eb-48d0-90dd-5addd626fc4e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.310064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e24c8-14eb-48d0-90dd-5addd626fc4e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.310064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e24c8-14eb-48d0-90dd-5addd626fc4e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.310064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 81/180] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:25.310064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e24c8-14eb-48d0-90dd-5addd626fc4e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:25.350392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66836ac9-cee7-421a-a6ca-3ec8f37e6a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:25.351623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66836ac9-cee7-421a-a6ca-3ec8f37e6a69 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.352356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66836ac9-cee7-421a-a6ca-3ec8f37e6a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.353752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66836ac9-cee7-421a-a6ca-3ec8f37e6a69 None admin] PATH_INFO: `/v3/projects/94ce177c584046218878ea073342cd16` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.355220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66836ac9-cee7-421a-a6ca-3ec8f37e6a69 None admin] RBAC: Authorizing `identity:get_project(project_id=94ce177c584046218878ea073342cd16)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:25.356910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66836ac9-cee7-421a-a6ca-3ec8f37e6a69 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:25.360066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 100/181] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:25 2022] GET /identity/v3/projects/94ce177c584046218878ea073342cd16 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:25.361233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66836ac9-cee7-421a-a6ca-3ec8f37e6a69 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:25.364888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:25.438596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e24c8-14eb-48d0-90dd-5addd626fc4e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:25.459462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:25.477416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64beb71-56d8-4643-9fba-fa0f0badae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:25.477904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64beb71-56d8-4643-9fba-fa0f0badae10 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.478241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64beb71-56d8-4643-9fba-fa0f0badae10 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.478524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64beb71-56d8-4643-9fba-fa0f0badae10 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.479388 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64beb71-56d8-4643-9fba-fa0f0badae10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:25.479689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64beb71-56d8-4643-9fba-fa0f0badae10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:25.481061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64beb71-56d8-4643-9fba-fa0f0badae10 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:25.486625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66836ac9-cee7-421a-a6ca-3ec8f37e6a69 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:25.490419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 82/182] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:25.491669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea4c5-7e5f-4256-90a1-417c6452f2b8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.491960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea4c5-7e5f-4256-90a1-417c6452f2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.492243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea4c5-7e5f-4256-90a1-417c6452f2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.510904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-be3ea4c5-7e5f-4256-90a1-417c6452f2b8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:25.512092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-be3ea4c5-7e5f-4256-90a1-417c6452f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:25.524262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ade48e-1ea1-431e-94b3-f0c0d4c82557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:25.525106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ade48e-1ea1-431e-94b3-f0c0d4c82557 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.525386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ade48e-1ea1-431e-94b3-f0c0d4c82557 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.525659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ade48e-1ea1-431e-94b3-f0c0d4c82557 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.526855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ade48e-1ea1-431e-94b3-f0c0d4c82557 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:25.528299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ade48e-1ea1-431e-94b3-f0c0d4c82557 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:25.534594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 101/183] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:25 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:25.534594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ade48e-1ea1-431e-94b3-f0c0d4c82557 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:25.588073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 83/184] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:25.599420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74ef28-95ab-47f6-af79-75185a026a5b None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.599707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74ef28-95ab-47f6-af79-75185a026a5b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.599998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74ef28-95ab-47f6-af79-75185a026a5b None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.601297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 84/185] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:25 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:25.601998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74ef28-95ab-47f6-af79-75185a026a5b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:25.638403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ade48e-1ea1-431e-94b3-f0c0d4c82557 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:25.649789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:25.678520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1dca8-b0fe-45f9-8e4a-59572cf2c7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:25.679427 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1dca8-b0fe-45f9-8e4a-59572cf2c7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.679705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1dca8-b0fe-45f9-8e4a-59572cf2c7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.679981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1dca8-b0fe-45f9-8e4a-59572cf2c7c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.680820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1dca8-b0fe-45f9-8e4a-59572cf2c7c3 None admin] RBAC: Adding query filter params (role.id=a70b63ede7014fa9bbcb4e03275a2664, scope.project.id=94ce177c584046218878ea073342cd16, user.id=654695f326044efdb73fe2956b7ce735) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:25.681133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1dca8-b0fe-45f9-8e4a-59572cf2c7c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:25.682624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1dca8-b0fe-45f9-8e4a-59572cf2c7c3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:25.689665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 102/186] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:25 2022] GET /identity/v3/role_assignments?user.id=654695f326044efdb73fe2956b7ce735&scope.project.id=94ce177c584046218878ea073342cd16&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:25.690350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1dca8-b0fe-45f9-8e4a-59572cf2c7c3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:25.703079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:25.738789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74ef28-95ab-47f6-af79-75185a026a5b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:25.779256 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78702452-582f-4377-bff1-1d50f9b4c850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:25.780149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78702452-582f-4377-bff1-1d50f9b4c850 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.780439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78702452-582f-4377-bff1-1d50f9b4c850 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.780693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78702452-582f-4377-bff1-1d50f9b4c850 None admin] PATH_INFO: `/v3/users/b71baf76a6154ab2b4134c4509189ddc` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.781925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78702452-582f-4377-bff1-1d50f9b4c850 None admin] RBAC: Authorizing `identity:get_user(user_id=b71baf76a6154ab2b4134c4509189ddc)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:25.783445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78702452-582f-4377-bff1-1d50f9b4c850 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:25.785769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 85/187] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:25 2022] GET /identity/v3/users/b71baf76a6154ab2b4134c4509189ddc => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:25.785769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263b750-7855-4cc6-9bcf-7c78020756e0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.787413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263b750-7855-4cc6-9bcf-7c78020756e0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.787608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263b750-7855-4cc6-9bcf-7c78020756e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.812063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b263b750-7855-4cc6-9bcf-7c78020756e0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:25.813288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b263b750-7855-4cc6-9bcf-7c78020756e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:25.823777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1dca8-b0fe-45f9-8e4a-59572cf2c7c3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:25.862438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b938669-b458-496c-9c32-04a07aa16078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:25.862868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b938669-b458-496c-9c32-04a07aa16078 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.863309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b938669-b458-496c-9c32-04a07aa16078 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.863586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b938669-b458-496c-9c32-04a07aa16078 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.864580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b938669-b458-496c-9c32-04a07aa16078 None admin] RBAC: Adding query filter params (role.id=120eeffad7094e10a69166d19aaa6439, scope.project.id=245d48325d884a83b0a57436df3e1418, user.id=654695f326044efdb73fe2956b7ce735) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:25.864900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b938669-b458-496c-9c32-04a07aa16078 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:25.866388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b938669-b458-496c-9c32-04a07aa16078 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:25.872659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 103/188] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:25 2022] GET /identity/v3/role_assignments?user.id=654695f326044efdb73fe2956b7ce735&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=120eeffad7094e10a69166d19aaa6439 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:25.873459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b938669-b458-496c-9c32-04a07aa16078 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:25.912947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 86/189] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:25.914635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f345be-56ab-4d4b-851b-d643408d94b5 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:25.914919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f345be-56ab-4d4b-851b-d643408d94b5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:25.915192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f345be-56ab-4d4b-851b-d643408d94b5 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:25.916505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 87/190] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:25.921043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f345be-56ab-4d4b-851b-d643408d94b5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:25.970756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:25.982671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:26.016772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b938669-b458-496c-9c32-04a07aa16078 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.049392 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f345be-56ab-4d4b-851b-d643408d94b5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.057376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc8e5a-94b0-41b2-8082-3b21d17be07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:26.058216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8e5a-94b0-41b2-8082-3b21d17be07a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.058868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8e5a-94b0-41b2-8082-3b21d17be07a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.058868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8e5a-94b0-41b2-8082-3b21d17be07a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.070724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc8e5a-94b0-41b2-8082-3b21d17be07a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7cfc8e5a-94b0-41b2-8082-3b21d17be07a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:26.078729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:26.079786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:26.080706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 104/191] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:26.081361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7a303-1d77-4350-af16-64b71fb1d2d1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.081762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7a303-1d77-4350-af16-64b71fb1d2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.082587 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7a303-1d77-4350-af16-64b71fb1d2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.091759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d08727-2362-46cd-8d6c-33adb1a378ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:26.092605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d08727-2362-46cd-8d6c-33adb1a378ba None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.092888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d08727-2362-46cd-8d6c-33adb1a378ba None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.093651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d08727-2362-46cd-8d6c-33adb1a378ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.101581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-edc7a303-1d77-4350-af16-64b71fb1d2d1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:26.102872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-edc7a303-1d77-4350-af16-64b71fb1d2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:26.164290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d08727-2362-46cd-8d6c-33adb1a378ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:26.165799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d08727-2362-46cd-8d6c-33adb1a378ba None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:26.175101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 105/192] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:26.176284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7a303-1d77-4350-af16-64b71fb1d2d1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:26.202513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:26.242395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:26.267761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 88/193] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:26.267761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d08727-2362-46cd-8d6c-33adb1a378ba None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:26.282978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7a303-1d77-4350-af16-64b71fb1d2d1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.317558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad815c4-20bf-4ce2-bd5e-18dc549a43de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:26.318387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad815c4-20bf-4ce2-bd5e-18dc549a43de None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.318669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad815c4-20bf-4ce2-bd5e-18dc549a43de None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.318942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad815c4-20bf-4ce2-bd5e-18dc549a43de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.329110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad815c4-20bf-4ce2-bd5e-18dc549a43de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3ad815c4-20bf-4ce2-bd5e-18dc549a43de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:26.337351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:26.338271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:26.339069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 106/194] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:26.339754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad815c4-20bf-4ce2-bd5e-18dc549a43de None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:26.372091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d08727-2362-46cd-8d6c-33adb1a378ba None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.404795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d08544-cfd7-4f09-b58b-1fef363a11e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:26.405454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d08544-cfd7-4f09-b58b-1fef363a11e4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.405737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d08544-cfd7-4f09-b58b-1fef363a11e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.406014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d08544-cfd7-4f09-b58b-1fef363a11e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.406780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d08544-cfd7-4f09-b58b-1fef363a11e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:26.407090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d08544-cfd7-4f09-b58b-1fef363a11e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:26.408455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d08544-cfd7-4f09-b58b-1fef363a11e4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:26.409751 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:26.417925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 89/195] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:26.418753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d08544-cfd7-4f09-b58b-1fef363a11e4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:26.446561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad815c4-20bf-4ce2-bd5e-18dc549a43de None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.466372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:26.480470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fd865-4f62-463d-a713-128467cb1fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:26.481263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fd865-4f62-463d-a713-128467cb1fce None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.481554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fd865-4f62-463d-a713-128467cb1fce None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.481848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fd865-4f62-463d-a713-128467cb1fce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.483028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fd865-4f62-463d-a713-128467cb1fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:26.484482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fd865-4f62-463d-a713-128467cb1fce None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:26.486123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 107/196] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:26.486885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fd865-4f62-463d-a713-128467cb1fce None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:26.520090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d08544-cfd7-4f09-b58b-1fef363a11e4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.556027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6984e1-8a8c-478f-9660-9ab92406620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:26.556878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6984e1-8a8c-478f-9660-9ab92406620a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.557159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6984e1-8a8c-478f-9660-9ab92406620a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.557430 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6984e1-8a8c-478f-9660-9ab92406620a None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.558654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6984e1-8a8c-478f-9660-9ab92406620a None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:26.560088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6984e1-8a8c-478f-9660-9ab92406620a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:26.561771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 90/197] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:26 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:26.562558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6984e1-8a8c-478f-9660-9ab92406620a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:26.587231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fd865-4f62-463d-a713-128467cb1fce None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.620246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac4bbe-6645-43f0-aec4-4442ffbd4b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:26.620899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac4bbe-6645-43f0-aec4-4442ffbd4b1d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.621195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac4bbe-6645-43f0-aec4-4442ffbd4b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.621453 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac4bbe-6645-43f0-aec4-4442ffbd4b1d None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.622228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac4bbe-6645-43f0-aec4-4442ffbd4b1d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:26.622534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac4bbe-6645-43f0-aec4-4442ffbd4b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:26.623904 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac4bbe-6645-43f0-aec4-4442ffbd4b1d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:26.626504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:26.632118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 108/198] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:26.632118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac4bbe-6645-43f0-aec4-4442ffbd4b1d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:26.664381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6984e1-8a8c-478f-9660-9ab92406620a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.686235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:26.698200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94dffe-ae72-44d3-9f23-9b72cd77afa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:26.699075 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94dffe-ae72-44d3-9f23-9b72cd77afa7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.699356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94dffe-ae72-44d3-9f23-9b72cd77afa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.699632 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94dffe-ae72-44d3-9f23-9b72cd77afa7 None admin] PATH_INFO: `/v3/users/6c14e51d5313465884069cd6b2d4f771` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.712680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94dffe-ae72-44d3-9f23-9b72cd77afa7 None admin] RBAC: Authorizing `identity:get_user(user_id=6c14e51d5313465884069cd6b2d4f771)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:26.714189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94dffe-ae72-44d3-9f23-9b72cd77afa7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:26.720142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 91/199] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:26 2022] GET /identity/v3/users/6c14e51d5313465884069cd6b2d4f771 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:26.720142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94dffe-ae72-44d3-9f23-9b72cd77afa7 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:26.736223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac4bbe-6645-43f0-aec4-4442ffbd4b1d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.770733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779d307-5cc5-466e-9700-921dc73af901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:26.771362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779d307-5cc5-466e-9700-921dc73af901 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.771649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779d307-5cc5-466e-9700-921dc73af901 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.771929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779d307-5cc5-466e-9700-921dc73af901 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.772754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779d307-5cc5-466e-9700-921dc73af901 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:26.774112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779d307-5cc5-466e-9700-921dc73af901 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:26.815999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94dffe-ae72-44d3-9f23-9b72cd77afa7 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.838368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:26.849391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d267d60-71a4-46ff-8687-51b42301c27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:26.850043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d267d60-71a4-46ff-8687-51b42301c27f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:26.850363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d267d60-71a4-46ff-8687-51b42301c27f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:26.850628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d267d60-71a4-46ff-8687-51b42301c27f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:26.851450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d267d60-71a4-46ff-8687-51b42301c27f None admin] RBAC: Adding query filter params (role.id=43bec89a20b54bdeb265c47e7af8d7b6, scope.project.id=245d48325d884a83b0a57436df3e1418, user.id=b71baf76a6154ab2b4134c4509189ddc) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:26.851759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d267d60-71a4-46ff-8687-51b42301c27f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:26.853196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d267d60-71a4-46ff-8687-51b42301c27f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:26.854578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 109/200] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:47:26 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:26.855395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779d307-5cc5-466e-9700-921dc73af901 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:26.869868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 92/201] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:26 2022] GET /identity/v3/role_assignments?user.id=b71baf76a6154ab2b4134c4509189ddc&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=43bec89a20b54bdeb265c47e7af8d7b6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:26.871126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d267d60-71a4-46ff-8687-51b42301c27f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:26.985235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d267d60-71a4-46ff-8687-51b42301c27f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:26.993710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779d307-5cc5-466e-9700-921dc73af901 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:27.005028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:27.028868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92a6cf-313a-4b25-bd9a-87a288e7d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:27.030223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92a6cf-313a-4b25-bd9a-87a288e7d44a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:27.030723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92a6cf-313a-4b25-bd9a-87a288e7d44a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:27.031354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92a6cf-313a-4b25-bd9a-87a288e7d44a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:27.041634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10debd13-e294-4c53-9063-73be4eb3e56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:27.042590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10debd13-e294-4c53-9063-73be4eb3e56b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:27.042878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10debd13-e294-4c53-9063-73be4eb3e56b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:27.043161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10debd13-e294-4c53-9063-73be4eb3e56b None admin] PATH_INFO: `/v3/users/6c14e51d5313465884069cd6b2d4f771` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:27.044580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10debd13-e294-4c53-9063-73be4eb3e56b None admin] RBAC: Authorizing `identity:get_user(user_id=6c14e51d5313465884069cd6b2d4f771)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:27.047365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10debd13-e294-4c53-9063-73be4eb3e56b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:27.055447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 110/202] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:26 2022] GET /identity/v3/users/6c14e51d5313465884069cd6b2d4f771 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:27.057455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10debd13-e294-4c53-9063-73be4eb3e56b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:27.119353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:27.137665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92a6cf-313a-4b25-bd9a-87a288e7d44a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:27.139253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92a6cf-313a-4b25-bd9a-87a288e7d44a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:27.190592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10debd13-e294-4c53-9063-73be4eb3e56b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:27.232941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76740453-01e1-4a9f-b170-af33dd0d9995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:27.233921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76740453-01e1-4a9f-b170-af33dd0d9995 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:27.234242 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76740453-01e1-4a9f-b170-af33dd0d9995 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:27.234522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76740453-01e1-4a9f-b170-af33dd0d9995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:27.259631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 93/203] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:27.266363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92a6cf-313a-4b25-bd9a-87a288e7d44a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:27.288538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:27.319582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76740453-01e1-4a9f-b170-af33dd0d9995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:27.320838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76740453-01e1-4a9f-b170-af33dd0d9995 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:27.375450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:27.396432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92a6cf-313a-4b25-bd9a-87a288e7d44a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:27.441015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f99c6-1582-4980-937f-5b605e37743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:27.442777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f99c6-1582-4980-937f-5b605e37743e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:27.443064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f99c6-1582-4980-937f-5b605e37743e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:27.443754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f99c6-1582-4980-937f-5b605e37743e None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:27.445209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f99c6-1582-4980-937f-5b605e37743e None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:27.446709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f99c6-1582-4980-937f-5b605e37743e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:27.447444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 111/204] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:27.453320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 94/205] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:27 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:27.459233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76740453-01e1-4a9f-b170-af33dd0d9995 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:27.459373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f99c6-1582-4980-937f-5b605e37743e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:27.555190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:27.586552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76740453-01e1-4a9f-b170-af33dd0d9995 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:27.598667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f99c6-1582-4980-937f-5b605e37743e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:27.630665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3d30d-6478-477e-a7f4-b776250081a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:27.631685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3d30d-6478-477e-a7f4-b776250081a8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:27.631966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3d30d-6478-477e-a7f4-b776250081a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:27.632240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3d30d-6478-477e-a7f4-b776250081a8 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:27.634115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3d30d-6478-477e-a7f4-b776250081a8 None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:27.635775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3d30d-6478-477e-a7f4-b776250081a8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:27.638263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 112/206] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:27 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:27.643870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3d30d-6478-477e-a7f4-b776250081a8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:27.649612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94791ad-bee8-43a2-9b76-9759d3cc1990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:27.650606 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94791ad-bee8-43a2-9b76-9759d3cc1990 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:27.651091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94791ad-bee8-43a2-9b76-9759d3cc1990 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:27.651564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94791ad-bee8-43a2-9b76-9759d3cc1990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:27.653106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94791ad-bee8-43a2-9b76-9759d3cc1990 None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=35d52ce6b9e5491abc31216be989797e, user.id=6c14e51d5313465884069cd6b2d4f771) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:27.653657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94791ad-bee8-43a2-9b76-9759d3cc1990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:27.655899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94791ad-bee8-43a2-9b76-9759d3cc1990 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:27.664566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 95/207] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:27 2022] GET /identity/v3/role_assignments?user.id=6c14e51d5313465884069cd6b2d4f771&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:27.774526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3d30d-6478-477e-a7f4-b776250081a8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:27.816481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb9d92-f317-40ff-a27a-96859e5260bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:27.817490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb9d92-f317-40ff-a27a-96859e5260bb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:27.817975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb9d92-f317-40ff-a27a-96859e5260bb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:27.818496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb9d92-f317-40ff-a27a-96859e5260bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:27.819532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb9d92-f317-40ff-a27a-96859e5260bb None admin] RBAC: Adding query filter params (role.id=120eeffad7094e10a69166d19aaa6439, scope.project.id=35d52ce6b9e5491abc31216be989797e, user.id=6c14e51d5313465884069cd6b2d4f771) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:27.820060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb9d92-f317-40ff-a27a-96859e5260bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:27.821736 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb9d92-f317-40ff-a27a-96859e5260bb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:27.828200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 113/208] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:27 2022] GET /identity/v3/role_assignments?user.id=6c14e51d5313465884069cd6b2d4f771&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=120eeffad7094e10a69166d19aaa6439 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:29.143026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ce684-90da-4655-ae20-4a7eb57a8fbe None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.143535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ce684-90da-4655-ae20-4a7eb57a8fbe None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.143607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ce684-90da-4655-ae20-4a7eb57a8fbe None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.145854 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 96/209] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:29.151286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7d628-41f3-4e28-8a39-178207ecfe34 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.152084 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7d628-41f3-4e28-8a39-178207ecfe34 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.152639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7d628-41f3-4e28-8a39-178207ecfe34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.165503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:29.172090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b2f7d628-41f3-4e28-8a39-178207ecfe34 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:29.173471 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b2f7d628-41f3-4e28-8a39-178207ecfe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:29.203477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb58a5c-394c-4993-9228-4653bc004541 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.203772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb58a5c-394c-4993-9228-4653bc004541 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.204044 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb58a5c-394c-4993-9228-4653bc004541 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.205649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 97/210] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:29.211047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa546690-ec64-4f8f-b9fa-057b1bc39bf7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.211336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa546690-ec64-4f8f-b9fa-057b1bc39bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.211614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa546690-ec64-4f8f-b9fa-057b1bc39bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.230060 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-aa546690-ec64-4f8f-b9fa-057b1bc39bf7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:29.231384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-aa546690-ec64-4f8f-b9fa-057b1bc39bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:29.245680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 114/211] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:29.250688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:29.274791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7d628-41f3-4e28-8a39-178207ecfe34 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:29.318298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 98/212] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:29.336942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa546690-ec64-4f8f-b9fa-057b1bc39bf7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:29.392540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7d628-41f3-4e28-8a39-178207ecfe34 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:29.430971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f38ab-ec10-45f6-8e2b-acea954baa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:29.431995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f38ab-ec10-45f6-8e2b-acea954baa3a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.432419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f38ab-ec10-45f6-8e2b-acea954baa3a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.432857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f38ab-ec10-45f6-8e2b-acea954baa3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.435887 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:29.445521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f38ab-ec10-45f6-8e2b-acea954baa3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:29.447726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa546690-ec64-4f8f-b9fa-057b1bc39bf7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c20f38ab-ec10-45f6-8e2b-acea954baa3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:29.453903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:29.456131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:29.457570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:29.458575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 115/213] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:29.465999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f38ab-ec10-45f6-8e2b-acea954baa3a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:29.492180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb24ad-44cd-46ff-b8e4-d0773c5a2b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:29.492981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb24ad-44cd-46ff-b8e4-d0773c5a2b32 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.493262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb24ad-44cd-46ff-b8e4-d0773c5a2b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.493536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb24ad-44cd-46ff-b8e4-d0773c5a2b32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.494725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb24ad-44cd-46ff-b8e4-d0773c5a2b32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:29.496165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb24ad-44cd-46ff-b8e4-d0773c5a2b32 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:29.502068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 99/214] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:29.503303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb24ad-44cd-46ff-b8e4-d0773c5a2b32 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:29.523028 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:29.599105 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f38ab-ec10-45f6-8e2b-acea954baa3a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:29.613458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb24ad-44cd-46ff-b8e4-d0773c5a2b32 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:29.639687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b107a-99aa-4b39-ba87-6536cc18b1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:29.640347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b107a-99aa-4b39-ba87-6536cc18b1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.640643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b107a-99aa-4b39-ba87-6536cc18b1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.641231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b107a-99aa-4b39-ba87-6536cc18b1b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.642334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b107a-99aa-4b39-ba87-6536cc18b1b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:29.642649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b107a-99aa-4b39-ba87-6536cc18b1b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:29.645197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b107a-99aa-4b39-ba87-6536cc18b1b2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:29.662137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c39e2-953c-40cd-968d-2859861aef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:29.662137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c39e2-953c-40cd-968d-2859861aef20 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.662137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c39e2-953c-40cd-968d-2859861aef20 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.662137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c39e2-953c-40cd-968d-2859861aef20 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.662492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c39e2-953c-40cd-968d-2859861aef20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:29.662589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 116/215] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:29.662924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c39e2-953c-40cd-968d-2859861aef20 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:29.664847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b107a-99aa-4b39-ba87-6536cc18b1b2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:29.730211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:29.774903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b107a-99aa-4b39-ba87-6536cc18b1b2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:29.790413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 100/216] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:47:29 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:47:29.819072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eaebd-74fc-4999-9832-b7873ac3936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:29.820339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eaebd-74fc-4999-9832-b7873ac3936e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.820826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eaebd-74fc-4999-9832-b7873ac3936e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.821297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eaebd-74fc-4999-9832-b7873ac3936e None admin] PATH_INFO: `/v3/users/aedb73c4ce524c8091e33a9161a50b18` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.821426 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc4a41-7e4a-4e9c-8cab-9a72aaf996c5 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.821680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc4a41-7e4a-4e9c-8cab-9a72aaf996c5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.822218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc4a41-7e4a-4e9c-8cab-9a72aaf996c5 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.827953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 101/217] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:47:29.829779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c41db-5aba-423f-8376-343c017ecd54 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.830211 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c41db-5aba-423f-8376-343c017ecd54 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.830588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c41db-5aba-423f-8376-343c017ecd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.840099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eaebd-74fc-4999-9832-b7873ac3936e None admin] RBAC: Authorizing `identity:get_user(user_id=aedb73c4ce524c8091e33a9161a50b18)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:29.841197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eaebd-74fc-4999-9832-b7873ac3936e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:29.843307 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 117/218] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:29 2022] GET /identity/v3/users/aedb73c4ce524c8091e33a9161a50b18 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:29.849212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75000ec0-b001-4f41-841a-23f2cbcaceeb None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.849560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75000ec0-b001-4f41-841a-23f2cbcaceeb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.849775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75000ec0-b001-4f41-841a-23f2cbcaceeb None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.851380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 118/219] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:29.851880 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75000ec0-b001-4f41-841a-23f2cbcaceeb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:29.853267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-766c41db-5aba-423f-8376-343c017ecd54 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:29.854547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-766c41db-5aba-423f-8376-343c017ecd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:29.933266 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 102/220] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:29.942220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42895500-ab90-41f8-8d54-3b0d0a826f1d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:29.942220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42895500-ab90-41f8-8d54-3b0d0a826f1d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:29.942220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42895500-ab90-41f8-8d54-3b0d0a826f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:29.942220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:29.956425 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-42895500-ab90-41f8-8d54-3b0d0a826f1d None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:29.958208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-42895500-ab90-41f8-8d54-3b0d0a826f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:29.979817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75000ec0-b001-4f41-841a-23f2cbcaceeb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:29.996751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:30.016170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be6733-191c-46cc-abd8-4ef324f98f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:30.016861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be6733-191c-46cc-abd8-4ef324f98f4c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:30.017106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be6733-191c-46cc-abd8-4ef324f98f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:30.018144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be6733-191c-46cc-abd8-4ef324f98f4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:30.035710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 103/221] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:30.036580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42895500-ab90-41f8-8d54-3b0d0a826f1d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:30.106140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be6733-191c-46cc-abd8-4ef324f98f4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:30.107680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be6733-191c-46cc-abd8-4ef324f98f4c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:30.167058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42895500-ab90-41f8-8d54-3b0d0a826f1d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:30.205703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:30.211658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3958798-c106-4adf-bead-c884bce0829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:30.212763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3958798-c106-4adf-bead-c884bce0829a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:30.213047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3958798-c106-4adf-bead-c884bce0829a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:30.213321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3958798-c106-4adf-bead-c884bce0829a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:30.227727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3958798-c106-4adf-bead-c884bce0829a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:30.230178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 119/222] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:30.231013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be6733-191c-46cc-abd8-4ef324f98f4c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d3958798-c106-4adf-bead-c884bce0829a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:30.236805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:30.237718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:30.239264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 104/223] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:30.240376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3958798-c106-4adf-bead-c884bce0829a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:30.259588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:30.371010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be6733-191c-46cc-abd8-4ef324f98f4c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:30.384699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3958798-c106-4adf-bead-c884bce0829a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:30.417078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c0ba8-a188-43be-a375-07ce77d96c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:30.418767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c0ba8-a188-43be-a375-07ce77d96c33 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:30.419543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c0ba8-a188-43be-a375-07ce77d96c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:30.420015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c0ba8-a188-43be-a375-07ce77d96c33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:30.427386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c42a9-133f-47c2-aa12-13dceab5c6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:30.428252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c42a9-133f-47c2-aa12-13dceab5c6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:30.428527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c42a9-133f-47c2-aa12-13dceab5c6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:30.428796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c42a9-133f-47c2-aa12-13dceab5c6b0 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:30.429985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c42a9-133f-47c2-aa12-13dceab5c6b0 None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:30.431960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c42a9-133f-47c2-aa12-13dceab5c6b0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:30.434391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 105/224] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:30 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:30.434786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c0ba8-a188-43be-a375-07ce77d96c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:30.435234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c42a9-133f-47c2-aa12-13dceab5c6b0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f06c0ba8-a188-43be-a375-07ce77d96c33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:30.445979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:30.447130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:30.447940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:30.451400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 120/225] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:30.452591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c0ba8-a188-43be-a375-07ce77d96c33 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:30.517566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:30.554265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:30.571629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c42a9-133f-47c2-aa12-13dceab5c6b0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:30.586826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c0ba8-a188-43be-a375-07ce77d96c33 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:30.616358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6036391-d99a-4a98-b555-2162b9dc45c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:30.617511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6036391-d99a-4a98-b555-2162b9dc45c1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:30.618119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6036391-d99a-4a98-b555-2162b9dc45c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:30.618594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6036391-d99a-4a98-b555-2162b9dc45c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:30.619532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6036391-d99a-4a98-b555-2162b9dc45c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:30.620045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6036391-d99a-4a98-b555-2162b9dc45c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:30.621840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6036391-d99a-4a98-b555-2162b9dc45c1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:30.629975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2d42-7026-4ad1-8ac5-cb05e29ca4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:30.630655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2d42-7026-4ad1-8ac5-cb05e29ca4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:30.630931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2d42-7026-4ad1-8ac5-cb05e29ca4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:30.631200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2d42-7026-4ad1-8ac5-cb05e29ca4d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:30.632028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e2d42-7026-4ad1-8ac5-cb05e29ca4d8 None admin] RBAC: Adding query filter params (role.id=a70b63ede7014fa9bbcb4e03275a2664, scope.project.id=35d52ce6b9e5491abc31216be989797e, user.id=aedb73c4ce524c8091e33a9161a50b18) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:30.632333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e2d42-7026-4ad1-8ac5-cb05e29ca4d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:30.633358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 106/226] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:30.633837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e2d42-7026-4ad1-8ac5-cb05e29ca4d8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:30.634607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6036391-d99a-4a98-b555-2162b9dc45c1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:30.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 121/227] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:30 2022] GET /identity/v3/role_assignments?user.id=aedb73c4ce524c8091e33a9161a50b18&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:30.640846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2d42-7026-4ad1-8ac5-cb05e29ca4d8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:30.750530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6036391-d99a-4a98-b555-2162b9dc45c1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:30.756135 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2d42-7026-4ad1-8ac5-cb05e29ca4d8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:30.777491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:30.790748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b85fb-c551-47b6-bc14-208341755386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:30.794118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b85fb-c551-47b6-bc14-208341755386 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:30.794118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b85fb-c551-47b6-bc14-208341755386 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:30.794118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b85fb-c551-47b6-bc14-208341755386 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:30.794118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b85fb-c551-47b6-bc14-208341755386 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:30.794118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b85fb-c551-47b6-bc14-208341755386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:30.794464 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236fafed-909f-448a-88d7-7bcb45fb8b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:30.795169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fafed-909f-448a-88d7-7bcb45fb8b09 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:30.795454 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fafed-909f-448a-88d7-7bcb45fb8b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:30.795744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fafed-909f-448a-88d7-7bcb45fb8b09 None admin] PATH_INFO: `/v3/users/6c14e51d5313465884069cd6b2d4f771` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:30.796821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b85fb-c551-47b6-bc14-208341755386 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:30.797193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236fafed-909f-448a-88d7-7bcb45fb8b09 None admin] RBAC: Authorizing `identity:get_user(user_id=6c14e51d5313465884069cd6b2d4f771)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:30.798773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236fafed-909f-448a-88d7-7bcb45fb8b09 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:30.801272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 122/228] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:30 2022] GET /identity/v3/users/6c14e51d5313465884069cd6b2d4f771 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:30.807256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236fafed-909f-448a-88d7-7bcb45fb8b09 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:30.810067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 107/229] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:30.815889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b85fb-c551-47b6-bc14-208341755386 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:30.828475 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:30.929516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b85fb-c551-47b6-bc14-208341755386 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:30.969884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7dbc0-9459-43a7-9879-f97a749014be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:30.970833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7dbc0-9459-43a7-9879-f97a749014be None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:30.971120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7dbc0-9459-43a7-9879-f97a749014be None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:30.971395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7dbc0-9459-43a7-9879-f97a749014be None admin] PATH_INFO: `/v3/users/6c14e51d5313465884069cd6b2d4f771` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:30.972672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7dbc0-9459-43a7-9879-f97a749014be None admin] RBAC: Authorizing `identity:get_user(user_id=6c14e51d5313465884069cd6b2d4f771)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:30.974185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7dbc0-9459-43a7-9879-f97a749014be None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:30.975994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 108/230] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:30 2022] GET /identity/v3/users/6c14e51d5313465884069cd6b2d4f771 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:30.983375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7dbc0-9459-43a7-9879-f97a749014be None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:31.012103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236fafed-909f-448a-88d7-7bcb45fb8b09 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:31.086819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:31.088522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae42a0-e9b9-47c6-8c9e-7415337d70f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:31.089394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae42a0-e9b9-47c6-8c9e-7415337d70f2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:31.089664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae42a0-e9b9-47c6-8c9e-7415337d70f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:31.089933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae42a0-e9b9-47c6-8c9e-7415337d70f2 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:31.094249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae42a0-e9b9-47c6-8c9e-7415337d70f2 None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:31.097513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae42a0-e9b9-47c6-8c9e-7415337d70f2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:31.102993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 123/231] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:30 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:31.108799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae42a0-e9b9-47c6-8c9e-7415337d70f2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:31.158295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7dbc0-9459-43a7-9879-f97a749014be None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:31.208378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f57fe5-3f74-4748-b5bf-f91cc984e2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:31.209266 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f57fe5-3f74-4748-b5bf-f91cc984e2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:31.209550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f57fe5-3f74-4748-b5bf-f91cc984e2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:31.209827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f57fe5-3f74-4748-b5bf-f91cc984e2f9 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:31.211078 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f57fe5-3f74-4748-b5bf-f91cc984e2f9 None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:31.211573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:31.212710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f57fe5-3f74-4748-b5bf-f91cc984e2f9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:31.214532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 109/232] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:30 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:31.219979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f57fe5-3f74-4748-b5bf-f91cc984e2f9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:31.223094 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae42a0-e9b9-47c6-8c9e-7415337d70f2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:31.259309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cfff9-29ca-4a8d-8563-150f9cf3ed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:31.261266 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cfff9-29ca-4a8d-8563-150f9cf3ed7b None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:31.261747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cfff9-29ca-4a8d-8563-150f9cf3ed7b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:31.262320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cfff9-29ca-4a8d-8563-150f9cf3ed7b None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e/users/6c14e51d5313465884069cd6b2d4f771/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:31.264514 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cfff9-29ca-4a8d-8563-150f9cf3ed7b None admin] RBAC: Authorizing `identity:create_grant(project_id=35d52ce6b9e5491abc31216be989797e, user_id=6c14e51d5313465884069cd6b2d4f771, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:31.266415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cfff9-29ca-4a8d-8563-150f9cf3ed7b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:31.355781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 124/233] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:47:31 2022] PUT /identity/v3/projects/35d52ce6b9e5491abc31216be989797e/users/6c14e51d5313465884069cd6b2d4f771/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:31.364998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f57fe5-3f74-4748-b5bf-f91cc984e2f9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:31.394636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:31.407585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81716b7-42e4-4e47-b3e6-12bfe3442400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:31.408544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81716b7-42e4-4e47-b3e6-12bfe3442400 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:31.408920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81716b7-42e4-4e47-b3e6-12bfe3442400 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:31.413792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81716b7-42e4-4e47-b3e6-12bfe3442400 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e/users/6c14e51d5313465884069cd6b2d4f771/roles/120eeffad7094e10a69166d19aaa6439` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:31.413792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81716b7-42e4-4e47-b3e6-12bfe3442400 None admin] RBAC: Authorizing `identity:create_grant(project_id=35d52ce6b9e5491abc31216be989797e, user_id=6c14e51d5313465884069cd6b2d4f771, role_id=120eeffad7094e10a69166d19aaa6439)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:31.413792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81716b7-42e4-4e47-b3e6-12bfe3442400 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:31.547306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 110/234] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:47:31 2022] PUT /identity/v3/projects/35d52ce6b9e5491abc31216be989797e/users/6c14e51d5313465884069cd6b2d4f771/roles/120eeffad7094e10a69166d19aaa6439 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 08:47:32.343391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94f3eb-f7c0-47f6-9040-03e8c936f6ae None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:32.343923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94f3eb-f7c0-47f6-9040-03e8c936f6ae None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:32.343923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94f3eb-f7c0-47f6-9040-03e8c936f6ae None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:32.345569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 125/235] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:32.351105 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a330f4-0e7d-4016-9147-b5bcbf91439b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:32.351506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a330f4-0e7d-4016-9147-b5bcbf91439b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:32.351661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a330f4-0e7d-4016-9147-b5bcbf91439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:32.374284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-78a330f4-0e7d-4016-9147-b5bcbf91439b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:32.375528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-78a330f4-0e7d-4016-9147-b5bcbf91439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:32.421612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27162945-2da0-4078-b26e-63db02c06299 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:32.421925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27162945-2da0-4078-b26e-63db02c06299 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:32.422196 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27162945-2da0-4078-b26e-63db02c06299 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:32.423682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 126/236] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:32.429138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f7f22-6492-4ade-87c2-1f97e80bf67f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:32.429431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f7f22-6492-4ade-87c2-1f97e80bf67f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:32.429717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f7f22-6492-4ade-87c2-1f97e80bf67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:32.448404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0b9f7f22-6492-4ade-87c2-1f97e80bf67f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:32.449695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0b9f7f22-6492-4ade-87c2-1f97e80bf67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:32.477340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 111/237] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:32.532237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a330f4-0e7d-4016-9147-b5bcbf91439b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:32.562198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 127/238] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:32.599408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f7f22-6492-4ade-87c2-1f97e80bf67f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:32.714075 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a330f4-0e7d-4016-9147-b5bcbf91439b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:32.717506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:32.730357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:32.775501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6918e8-ee81-4ccb-8b67-bb35e3d95196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:32.776306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6918e8-ee81-4ccb-8b67-bb35e3d95196 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:32.776588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6918e8-ee81-4ccb-8b67-bb35e3d95196 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:32.776870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6918e8-ee81-4ccb-8b67-bb35e3d95196 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:32.789723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f7f22-6492-4ade-87c2-1f97e80bf67f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:32.806803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6918e8-ee81-4ccb-8b67-bb35e3d95196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-8d6918e8-ee81-4ccb-8b67-bb35e3d95196 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:32.814671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:32.815688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:32.817162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 112/239] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:32.824810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6918e8-ee81-4ccb-8b67-bb35e3d95196 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:32.863646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5209b47-a800-4f8b-9a23-c523a3a30fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:32.864451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5209b47-a800-4f8b-9a23-c523a3a30fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:32.864732 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5209b47-a800-4f8b-9a23-c523a3a30fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:32.865003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5209b47-a800-4f8b-9a23-c523a3a30fc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:32.867292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5209b47-a800-4f8b-9a23-c523a3a30fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:32.871392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5209b47-a800-4f8b-9a23-c523a3a30fc5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:32.873026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 128/240] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:32.880851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5209b47-a800-4f8b-9a23-c523a3a30fc5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:33.015394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6918e8-ee81-4ccb-8b67-bb35e3d95196 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:33.078976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5209b47-a800-4f8b-9a23-c523a3a30fc5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:33.085909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3cfff-8f77-4efa-8036-e9d298e7c082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:33.086634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3cfff-8f77-4efa-8036-e9d298e7c082 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.086912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3cfff-8f77-4efa-8036-e9d298e7c082 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.087185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3cfff-8f77-4efa-8036-e9d298e7c082 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.087949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3cfff-8f77-4efa-8036-e9d298e7c082 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:33.088259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3cfff-8f77-4efa-8036-e9d298e7c082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:33.089675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3cfff-8f77-4efa-8036-e9d298e7c082 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:33.111939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 113/241] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:33.112911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3cfff-8f77-4efa-8036-e9d298e7c082 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:33.131809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bdbf9-99ae-4dd9-aa0c-f589e6078c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:33.132411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bdbf9-99ae-4dd9-aa0c-f589e6078c6c None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.132696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bdbf9-99ae-4dd9-aa0c-f589e6078c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.132962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bdbf9-99ae-4dd9-aa0c-f589e6078c6c None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.133687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bdbf9-99ae-4dd9-aa0c-f589e6078c6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:33.135165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bdbf9-99ae-4dd9-aa0c-f589e6078c6c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:33.142182 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:33.161702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:33.227351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 129/242] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:47:32 2022] POST /identity/v3/users => generated 293 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:33.228671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657feec-4b2e-414c-882e-70b88e4d45ba None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.228961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657feec-4b2e-414c-882e-70b88e4d45ba None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.229237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657feec-4b2e-414c-882e-70b88e4d45ba None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.236687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 130/243] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:33.242533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3cfff-8f77-4efa-8036-e9d298e7c082 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:33.243040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef26cb8-3c71-4468-bfc3-9930c9a76017 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.243328 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef26cb8-3c71-4468-bfc3-9930c9a76017 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.243607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef26cb8-3c71-4468-bfc3-9930c9a76017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.287133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fef26cb8-3c71-4468-bfc3-9930c9a76017 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:33.287558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e147-d131-4578-9217-6ee56520a80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:33.288347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fef26cb8-3c71-4468-bfc3-9930c9a76017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:33.288851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e147-d131-4578-9217-6ee56520a80a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.289300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e147-d131-4578-9217-6ee56520a80a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.289745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e147-d131-4578-9217-6ee56520a80a None admin] PATH_INFO: `/v3/users/ea013e6108e2453cbba79a5bd941750d` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.307958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986e147-d131-4578-9217-6ee56520a80a None admin] RBAC: Authorizing `identity:get_user(user_id=ea013e6108e2453cbba79a5bd941750d)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:33.310556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986e147-d131-4578-9217-6ee56520a80a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:33.312860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 114/244] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:33 2022] GET /identity/v3/users/ea013e6108e2453cbba79a5bd941750d => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:33.319023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e147-d131-4578-9217-6ee56520a80a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:33.430234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:33.442958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e147-d131-4578-9217-6ee56520a80a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:33.486593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 131/245] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:33.488347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c76dd-db64-40ed-b430-a46246e22daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:33.489006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c76dd-db64-40ed-b430-a46246e22daf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.489288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c76dd-db64-40ed-b430-a46246e22daf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.489561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c76dd-db64-40ed-b430-a46246e22daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.516855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef26cb8-3c71-4468-bfc3-9930c9a76017 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:33.583421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c76dd-db64-40ed-b430-a46246e22daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:33.585781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c76dd-db64-40ed-b430-a46246e22daf None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:33.639265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef26cb8-3c71-4468-bfc3-9930c9a76017 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:33.668199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:33.689599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc7a7b-14bc-4ef1-8a7d-99b1a596ac7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:33.690472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc7a7b-14bc-4ef1-8a7d-99b1a596ac7d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.690754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc7a7b-14bc-4ef1-8a7d-99b1a596ac7d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.691246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc7a7b-14bc-4ef1-8a7d-99b1a596ac7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.705578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:33.706350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc7a7b-14bc-4ef1-8a7d-99b1a596ac7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:33.717492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 115/246] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-65fc7a7b-14bc-4ef1-8a7d-99b1a596ac7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:33.718364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:33.723261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:33.724121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 132/247] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:33.724789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c76dd-db64-40ed-b430-a46246e22daf None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:33.729519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc7a7b-14bc-4ef1-8a7d-99b1a596ac7d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:33.857315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c76dd-db64-40ed-b430-a46246e22daf None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:33.858294 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc7a7b-14bc-4ef1-8a7d-99b1a596ac7d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:33.895674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cfd89-e3dd-44cf-bca8-13fec59d02a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:33.896710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cfd89-e3dd-44cf-bca8-13fec59d02a6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.897279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cfd89-e3dd-44cf-bca8-13fec59d02a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.897697 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cfd89-e3dd-44cf-bca8-13fec59d02a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.898624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cfd89-e3dd-44cf-bca8-13fec59d02a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:33.899072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cfd89-e3dd-44cf-bca8-13fec59d02a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:33.901196 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cfd89-e3dd-44cf-bca8-13fec59d02a6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:33.901330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec702cd-a241-4d60-aa86-c8b745cf2cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:33.901999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec702cd-a241-4d60-aa86-c8b745cf2cfd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.902399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec702cd-a241-4d60-aa86-c8b745cf2cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.902765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec702cd-a241-4d60-aa86-c8b745cf2cfd None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.904075 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec702cd-a241-4d60-aa86-c8b745cf2cfd None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:33.906535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec702cd-a241-4d60-aa86-c8b745cf2cfd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:33.908555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 116/248] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:33 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:33.910551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b4d4c-93c4-40a9-aafa-d7e1d9620af2 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.910551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b4d4c-93c4-40a9-aafa-d7e1d9620af2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.910997 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b4d4c-93c4-40a9-aafa-d7e1d9620af2 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.916318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 117/249] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:33.916318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b4d4c-93c4-40a9-aafa-d7e1d9620af2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:33.917720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 133/250] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:33.919838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee568-6ab8-493f-afec-41f4984bf5a8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:33.920762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee568-6ab8-493f-afec-41f4984bf5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:33.921170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee568-6ab8-493f-afec-41f4984bf5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:33.950481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1c4ee568-6ab8-493f-afec-41f4984bf5a8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:33.952076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1c4ee568-6ab8-493f-afec-41f4984bf5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:33.991499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:34.012849 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:34.064156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b4d4c-93c4-40a9-aafa-d7e1d9620af2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:34.069048 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 134/251] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:34.076334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ee568-6ab8-493f-afec-41f4984bf5a8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:34.104121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58d9cf-1513-47d1-b4dd-26e27e0a9d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:34.104932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58d9cf-1513-47d1-b4dd-26e27e0a9d09 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:34.105219 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58d9cf-1513-47d1-b4dd-26e27e0a9d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:34.105486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58d9cf-1513-47d1-b4dd-26e27e0a9d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:34.106491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58d9cf-1513-47d1-b4dd-26e27e0a9d09 None admin] RBAC: Adding query filter params (role.id=43bec89a20b54bdeb265c47e7af8d7b6, scope.project.id=35d52ce6b9e5491abc31216be989797e, user.id=ea013e6108e2453cbba79a5bd941750d) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:34.106818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58d9cf-1513-47d1-b4dd-26e27e0a9d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:34.108626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58d9cf-1513-47d1-b4dd-26e27e0a9d09 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:34.115590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 118/252] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:33 2022] GET /identity/v3/role_assignments?user.id=ea013e6108e2453cbba79a5bd941750d&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=43bec89a20b54bdeb265c47e7af8d7b6 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:34.117103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5f8c3-92ae-4e27-a6f4-d76e61d9a170 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:34.117393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5f8c3-92ae-4e27-a6f4-d76e61d9a170 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:34.117842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5f8c3-92ae-4e27-a6f4-d76e61d9a170 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:34.119716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 119/253] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:34.125069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5f8c3-92ae-4e27-a6f4-d76e61d9a170 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:34.213647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ee568-6ab8-493f-afec-41f4984bf5a8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:34.260187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35faf6-b9e1-4890-9ac4-93877ffbcbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:34.267162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35faf6-b9e1-4890-9ac4-93877ffbcbfc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:34.267721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35faf6-b9e1-4890-9ac4-93877ffbcbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:34.268193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35faf6-b9e1-4890-9ac4-93877ffbcbfc None admin] PATH_INFO: `/v3/users/aedb73c4ce524c8091e33a9161a50b18` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:34.270358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35faf6-b9e1-4890-9ac4-93877ffbcbfc None admin] RBAC: Authorizing `identity:get_user(user_id=aedb73c4ce524c8091e33a9161a50b18)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:34.272083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35faf6-b9e1-4890-9ac4-93877ffbcbfc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:34.276842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 135/254] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:34 2022] GET /identity/v3/users/aedb73c4ce524c8091e33a9161a50b18 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:34.280323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2f66b-34c3-419d-bdf6-ef0cc10bd582 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:34.280930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2f66b-34c3-419d-bdf6-ef0cc10bd582 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:34.282359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2f66b-34c3-419d-bdf6-ef0cc10bd582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:34.313646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-99b2f66b-34c3-419d-bdf6-ef0cc10bd582 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:34.315152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-99b2f66b-34c3-419d-bdf6-ef0cc10bd582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:34.325271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5f8c3-92ae-4e27-a6f4-d76e61d9a170 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:34.366339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:34.371654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef9223-3d67-40e8-8aeb-c2eb7bdeb74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:34.372472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef9223-3d67-40e8-8aeb-c2eb7bdeb74e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:34.372755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef9223-3d67-40e8-8aeb-c2eb7bdeb74e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:34.373028 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef9223-3d67-40e8-8aeb-c2eb7bdeb74e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:34.384147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:34.400452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef9223-3d67-40e8-8aeb-c2eb7bdeb74e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-51ef9223-3d67-40e8-8aeb-c2eb7bdeb74e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:34.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:34.409239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:34.410616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 120/255] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:34.427209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef9223-3d67-40e8-8aeb-c2eb7bdeb74e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:34.450688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 136/256] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:34.451530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2f66b-34c3-419d-bdf6-ef0cc10bd582 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:34.631286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef9223-3d67-40e8-8aeb-c2eb7bdeb74e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:34.642129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2f66b-34c3-419d-bdf6-ef0cc10bd582 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:34.686921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a9de-efb1-4a5a-a17a-2e6b83e1701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:34.688313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a9de-efb1-4a5a-a17a-2e6b83e1701a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:34.688807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a9de-efb1-4a5a-a17a-2e6b83e1701a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:34.689274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a9de-efb1-4a5a-a17a-2e6b83e1701a None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:34.690670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a9de-efb1-4a5a-a17a-2e6b83e1701a None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:34.693461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a9de-efb1-4a5a-a17a-2e6b83e1701a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:34.695392 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 121/257] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:34 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:34.696336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a9de-efb1-4a5a-a17a-2e6b83e1701a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:34.718746 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5df9a7-5df1-4817-bbc9-9cff33599175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:34.719429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df9a7-5df1-4817-bbc9-9cff33599175 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:34.719786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df9a7-5df1-4817-bbc9-9cff33599175 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:34.720063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df9a7-5df1-4817-bbc9-9cff33599175 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:34.720874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5df9a7-5df1-4817-bbc9-9cff33599175 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:34.721185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5df9a7-5df1-4817-bbc9-9cff33599175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:34.722852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5df9a7-5df1-4817-bbc9-9cff33599175 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:34.742295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 137/258] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:34.743355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5df9a7-5df1-4817-bbc9-9cff33599175 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:34.766291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:34.803833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:34.852591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a9de-efb1-4a5a-a17a-2e6b83e1701a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:34.917308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850942bb-e877-4d6f-b7c5-13277264f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:34.919099 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850942bb-e877-4d6f-b7c5-13277264f487 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:34.919738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850942bb-e877-4d6f-b7c5-13277264f487 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:34.921504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850942bb-e877-4d6f-b7c5-13277264f487 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:34.927299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5df9a7-5df1-4817-bbc9-9cff33599175 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:34.944776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850942bb-e877-4d6f-b7c5-13277264f487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-850942bb-e877-4d6f-b7c5-13277264f487 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:34.954468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:34.955338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:34.956134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 122/259] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:34.957437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850942bb-e877-4d6f-b7c5-13277264f487 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:35.006176 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b95c9-9084-450a-a07d-ddce93e40431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:35.008771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b95c9-9084-450a-a07d-ddce93e40431 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:35.010271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b95c9-9084-450a-a07d-ddce93e40431 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:35.010757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b95c9-9084-450a-a07d-ddce93e40431 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e/users/aedb73c4ce524c8091e33a9161a50b18/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:35.015671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b95c9-9084-450a-a07d-ddce93e40431 None admin] RBAC: Authorizing `identity:create_grant(project_id=35d52ce6b9e5491abc31216be989797e, user_id=aedb73c4ce524c8091e33a9161a50b18, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:35.021278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b95c9-9084-450a-a07d-ddce93e40431 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:35.104925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 138/260] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:47:34 2022] PUT /identity/v3/projects/35d52ce6b9e5491abc31216be989797e/users/aedb73c4ce524c8091e33a9161a50b18/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:35.106132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b95c9-9084-450a-a07d-ddce93e40431 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:35.124840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850942bb-e877-4d6f-b7c5-13277264f487 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:35.151707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:35.206790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ad8c8-5f4e-4145-9efb-ccfdafc25da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:35.208713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ad8c8-5f4e-4145-9efb-ccfdafc25da3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:35.210231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ad8c8-5f4e-4145-9efb-ccfdafc25da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:35.211607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ad8c8-5f4e-4145-9efb-ccfdafc25da3 None admin] PATH_INFO: `/v3/users/6c14e51d5313465884069cd6b2d4f771` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:35.213352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ad8c8-5f4e-4145-9efb-ccfdafc25da3 None admin] RBAC: Authorizing `identity:get_user(user_id=6c14e51d5313465884069cd6b2d4f771)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:35.216643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ad8c8-5f4e-4145-9efb-ccfdafc25da3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:35.219636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 123/261] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:34 2022] GET /identity/v3/users/6c14e51d5313465884069cd6b2d4f771 => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:35.231131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ad8c8-5f4e-4145-9efb-ccfdafc25da3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:35.353967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:35.381569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b95c9-9084-450a-a07d-ddce93e40431 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:35.458676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb5a48-2531-41a4-b739-75675d9cd442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:35.460133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb5a48-2531-41a4-b739-75675d9cd442 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:35.460772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb5a48-2531-41a4-b739-75675d9cd442 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:35.461389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb5a48-2531-41a4-b739-75675d9cd442 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:35.462535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb5a48-2531-41a4-b739-75675d9cd442 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:35.463248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb5a48-2531-41a4-b739-75675d9cd442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:35.464848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb5a48-2531-41a4-b739-75675d9cd442 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:35.475296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 139/262] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:35.489719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb5a48-2531-41a4-b739-75675d9cd442 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:35.497583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ad8c8-5f4e-4145-9efb-ccfdafc25da3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:35.536487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a19a1-2706-4adc-91ba-df906cd74c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:35.537221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a19a1-2706-4adc-91ba-df906cd74c07 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:35.537417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a19a1-2706-4adc-91ba-df906cd74c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:35.537697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a19a1-2706-4adc-91ba-df906cd74c07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:35.636004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:35.672715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb5a48-2531-41a4-b739-75675d9cd442 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:35.681503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a19a1-2706-4adc-91ba-df906cd74c07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:35.682989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a19a1-2706-4adc-91ba-df906cd74c07 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:35.766199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c9536-2414-4aca-b1d9-0b0f951409d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:35.767014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9536-2414-4aca-b1d9-0b0f951409d5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:35.767297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9536-2414-4aca-b1d9-0b0f951409d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:35.767576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c9536-2414-4aca-b1d9-0b0f951409d5 None admin] PATH_INFO: `/v3/users/6c14e51d5313465884069cd6b2d4f771` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:35.774815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c9536-2414-4aca-b1d9-0b0f951409d5 None admin] RBAC: Authorizing `identity:get_user(user_id=6c14e51d5313465884069cd6b2d4f771)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:35.776313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c9536-2414-4aca-b1d9-0b0f951409d5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:35.778345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 140/263] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:35 2022] GET /identity/v3/users/6c14e51d5313465884069cd6b2d4f771 => generated 283 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:35.796099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c9536-2414-4aca-b1d9-0b0f951409d5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:35.813259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 124/264] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:35.819992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a19a1-2706-4adc-91ba-df906cd74c07 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:35.867371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:35.912183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:35.936813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a19a1-2706-4adc-91ba-df906cd74c07 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:35.975789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343de031-bc29-434d-98ee-23c2fb710e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:35.976658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343de031-bc29-434d-98ee-23c2fb710e0a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:35.976938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343de031-bc29-434d-98ee-23c2fb710e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:35.977212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343de031-bc29-434d-98ee-23c2fb710e0a None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:35.979777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343de031-bc29-434d-98ee-23c2fb710e0a None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:35.980023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343de031-bc29-434d-98ee-23c2fb710e0a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:35.985890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 125/265] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:35 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:35.988813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343de031-bc29-434d-98ee-23c2fb710e0a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:36.030906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c9536-2414-4aca-b1d9-0b0f951409d5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:36.087733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51078f0-d250-4a8a-a3a6-5a605ba0c9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:36.088782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51078f0-d250-4a8a-a3a6-5a605ba0c9ce None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.089263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51078f0-d250-4a8a-a3a6-5a605ba0c9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.089731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51078f0-d250-4a8a-a3a6-5a605ba0c9ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.103051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343de031-bc29-434d-98ee-23c2fb710e0a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:36.156561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c697d-54e2-4e4d-8bc6-110e285e04c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:36.157229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c697d-54e2-4e4d-8bc6-110e285e04c5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.157571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c697d-54e2-4e4d-8bc6-110e285e04c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.157850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c697d-54e2-4e4d-8bc6-110e285e04c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.158962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c697d-54e2-4e4d-8bc6-110e285e04c5 None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=35d52ce6b9e5491abc31216be989797e, user.id=6c14e51d5313465884069cd6b2d4f771) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:36.159237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c697d-54e2-4e4d-8bc6-110e285e04c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:36.161777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c697d-54e2-4e4d-8bc6-110e285e04c5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:36.173639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51078f0-d250-4a8a-a3a6-5a605ba0c9ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:36.174250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 126/266] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:35 2022] GET /identity/v3/role_assignments?user.id=6c14e51d5313465884069cd6b2d4f771&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:36.175761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51078f0-d250-4a8a-a3a6-5a605ba0c9ce None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:36.205463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:36.208873 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b93cb6-b6b3-4776-a814-ca875e2f0d8e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.209374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b93cb6-b6b3-4776-a814-ca875e2f0d8e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.209860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b93cb6-b6b3-4776-a814-ca875e2f0d8e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.215305 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 127/267] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:36.217191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177e721-8fe5-4ee8-acc4-67a7a094bc09 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.220922 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177e721-8fe5-4ee8-acc4-67a7a094bc09 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.220922 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177e721-8fe5-4ee8-acc4-67a7a094bc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.265218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c177e721-8fe5-4ee8-acc4-67a7a094bc09 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:36.266484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c177e721-8fe5-4ee8-acc4-67a7a094bc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:36.285494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:36.392102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 141/268] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:36.399680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51078f0-d250-4a8a-a3a6-5a605ba0c9ce None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:36.446808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 128/269] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:36.446808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb56d2-0550-4ea3-873e-dce9254d3af3 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.446808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb56d2-0550-4ea3-873e-dce9254d3af3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.446808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb56d2-0550-4ea3-873e-dce9254d3af3 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.448397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 129/270] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:36.453563 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f970264-d17e-4f62-a3d4-2010e3f0e18c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.453846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f970264-d17e-4f62-a3d4-2010e3f0e18c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.454155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f970264-d17e-4f62-a3d4-2010e3f0e18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.472935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0f970264-d17e-4f62-a3d4-2010e3f0e18c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:36.473843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0f970264-d17e-4f62-a3d4-2010e3f0e18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:36.554523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51078f0-d250-4a8a-a3a6-5a605ba0c9ce None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:36.562669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 130/271] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:36.563930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f970264-d17e-4f62-a3d4-2010e3f0e18c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:36.590670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:36.611649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc12e51-4224-404d-a4d8-bc996ea97e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:36.613331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc12e51-4224-404d-a4d8-bc996ea97e94 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.613757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc12e51-4224-404d-a4d8-bc996ea97e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.614192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc12e51-4224-404d-a4d8-bc996ea97e94 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.617071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc12e51-4224-404d-a4d8-bc996ea97e94 None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:36.618746 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc12e51-4224-404d-a4d8-bc996ea97e94 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:36.620835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 142/272] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:36 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:36.624256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc12e51-4224-404d-a4d8-bc996ea97e94 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:36.655461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:36.698983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f970264-d17e-4f62-a3d4-2010e3f0e18c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:36.733388 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57fc57-9bc2-426e-a5cf-53ace66ca7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:36.734439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57fc57-9bc2-426e-a5cf-53ace66ca7ee None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.734714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57fc57-9bc2-426e-a5cf-53ace66ca7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.734987 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57fc57-9bc2-426e-a5cf-53ace66ca7ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.736127 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57fc57-9bc2-426e-a5cf-53ace66ca7ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:36.737569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57fc57-9bc2-426e-a5cf-53ace66ca7ee None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:36.739891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 131/273] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:36.742889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57fc57-9bc2-426e-a5cf-53ace66ca7ee None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:36.743102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc12e51-4224-404d-a4d8-bc996ea97e94 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:36.780867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2cf96-f94c-4987-8c9f-2582f0dc2810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:36.781895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2cf96-f94c-4987-8c9f-2582f0dc2810 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.782354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2cf96-f94c-4987-8c9f-2582f0dc2810 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.782761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2cf96-f94c-4987-8c9f-2582f0dc2810 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.794897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2cf96-f94c-4987-8c9f-2582f0dc2810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6ef2cf96-f94c-4987-8c9f-2582f0dc2810 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:36.804274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:36.805292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:36.806306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:36.808263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 143/274] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:36.809721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961b4c0-5237-41a3-a3fb-05bb947361ae None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.810167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961b4c0-5237-41a3-a3fb-05bb947361ae None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.810578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961b4c0-5237-41a3-a3fb-05bb947361ae None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.811963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 144/275] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:36.812818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961b4c0-5237-41a3-a3fb-05bb947361ae None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:36.857814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57fc57-9bc2-426e-a5cf-53ace66ca7ee None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:36.870158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:36.895092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6d51f-b532-426d-aa26-727f23cdc5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:36.896018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6d51f-b532-426d-aa26-727f23cdc5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.896501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6d51f-b532-426d-aa26-727f23cdc5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.896948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6d51f-b532-426d-aa26-727f23cdc5a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.897947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6d51f-b532-426d-aa26-727f23cdc5a4 None admin] RBAC: Adding query filter params (role.id=120eeffad7094e10a69166d19aaa6439, scope.project.id=35d52ce6b9e5491abc31216be989797e, user.id=6c14e51d5313465884069cd6b2d4f771) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:36.898537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6d51f-b532-426d-aa26-727f23cdc5a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:36.900142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6d51f-b532-426d-aa26-727f23cdc5a4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:36.907370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 132/276] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:36 2022] GET /identity/v3/role_assignments?user.id=6c14e51d5313465884069cd6b2d4f771&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=120eeffad7094e10a69166d19aaa6439 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:36.908455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6d51f-b532-426d-aa26-727f23cdc5a4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:36.912047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:36.941940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961b4c0-5237-41a3-a3fb-05bb947361ae None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:36.991349 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb11ecd-321b-414f-be11-de8eedec225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:36.992466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb11ecd-321b-414f-be11-de8eedec225c None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:36.992975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb11ecd-321b-414f-be11-de8eedec225c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:36.993447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb11ecd-321b-414f-be11-de8eedec225c None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:36.994508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb11ecd-321b-414f-be11-de8eedec225c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:36.995978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb11ecd-321b-414f-be11-de8eedec225c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:37.034309 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6d51f-b532-426d-aa26-727f23cdc5a4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:37.073669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff2f00-d8a4-408e-a16f-705b96861d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:37.074596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff2f00-d8a4-408e-a16f-705b96861d9a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:37.075020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff2f00-d8a4-408e-a16f-705b96861d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:37.075444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff2f00-d8a4-408e-a16f-705b96861d9a None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:37.076356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff2f00-d8a4-408e-a16f-705b96861d9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:37.076830 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff2f00-d8a4-408e-a16f-705b96861d9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:37.078242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff2f00-d8a4-408e-a16f-705b96861d9a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:37.090209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 133/277] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:37.091523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7ad5b-5468-4c8d-ba94-f6fad6bc4e2b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:37.091813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7ad5b-5468-4c8d-ba94-f6fad6bc4e2b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:37.092089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7ad5b-5468-4c8d-ba94-f6fad6bc4e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:37.109084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-67d7ad5b-5468-4c8d-ba94-f6fad6bc4e2b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:37.110305 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-67d7ad5b-5468-4c8d-ba94-f6fad6bc4e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:37.120707 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 145/278] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:47:36 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:37.121621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb11ecd-321b-414f-be11-de8eedec225c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:37.191650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:37.203166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 134/279] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:37.239284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7ad5b-5468-4c8d-ba94-f6fad6bc4e2b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:37.282370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb11ecd-321b-414f-be11-de8eedec225c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:37.292564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:37.333547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd7b5a-920f-43af-8778-60694a2c7fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:37.334386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7b5a-920f-43af-8778-60694a2c7fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:37.334692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7b5a-920f-43af-8778-60694a2c7fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:37.334934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7b5a-920f-43af-8778-60694a2c7fa8 None admin] PATH_INFO: `/v3/users/b7f15ae95c434a9d87f0ef44ecb3f009` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:37.361149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd7b5a-920f-43af-8778-60694a2c7fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=b7f15ae95c434a9d87f0ef44ecb3f009)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:37.362539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7ad5b-5468-4c8d-ba94-f6fad6bc4e2b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:37.363240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd7b5a-920f-43af-8778-60694a2c7fa8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:37.369833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 146/280] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:37 2022] GET /identity/v3/users/b7f15ae95c434a9d87f0ef44ecb3f009 => generated 293 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:37.375012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd7b5a-920f-43af-8778-60694a2c7fa8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:37.417316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fc4c5-ffa1-40e8-b200-d23f77977f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:37.418163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fc4c5-ffa1-40e8-b200-d23f77977f51 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:37.418452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fc4c5-ffa1-40e8-b200-d23f77977f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:37.418779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fc4c5-ffa1-40e8-b200-d23f77977f51 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:37.439532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fc4c5-ffa1-40e8-b200-d23f77977f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-2d1fc4c5-ffa1-40e8-b200-d23f77977f51 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:37.448831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:37.449763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:37.450611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 135/281] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:37.455910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fc4c5-ffa1-40e8-b200-d23f77977f51 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:37.515237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd7b5a-920f-43af-8778-60694a2c7fa8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:37.525758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:37.559353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df9ac5-851c-468d-bb07-65529b2f4f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:37.560291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df9ac5-851c-468d-bb07-65529b2f4f09 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:37.560571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df9ac5-851c-468d-bb07-65529b2f4f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:37.560842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df9ac5-851c-468d-bb07-65529b2f4f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:37.561674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df9ac5-851c-468d-bb07-65529b2f4f09 None admin] RBAC: Adding query filter params (role.id=a70b63ede7014fa9bbcb4e03275a2664, user.id=b7f15ae95c434a9d87f0ef44ecb3f009, scope.system=all) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:37.561976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df9ac5-851c-468d-bb07-65529b2f4f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:37.563947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df9ac5-851c-468d-bb07-65529b2f4f09 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:37.573250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 147/282] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 13 08:47:37 2022] GET /identity/v3/role_assignments?user.id=b7f15ae95c434a9d87f0ef44ecb3f009&scope.system=all&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:37.582875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fc4c5-ffa1-40e8-b200-d23f77977f51 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:37.623017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd3b1f-5602-4ed5-a265-33478187f7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:37.623705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd3b1f-5602-4ed5-a265-33478187f7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:37.623984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd3b1f-5602-4ed5-a265-33478187f7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:37.624259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd3b1f-5602-4ed5-a265-33478187f7e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:37.625018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd3b1f-5602-4ed5-a265-33478187f7e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:37.625324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd3b1f-5602-4ed5-a265-33478187f7e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:37.626758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd3b1f-5602-4ed5-a265-33478187f7e9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:37.637476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 136/283] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:37.643457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df9ac5-851c-468d-bb07-65529b2f4f09 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:37.683120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:37.785500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df9ac5-851c-468d-bb07-65529b2f4f09 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:37.825359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0107939-a8d4-46d4-9687-4c57b46d4d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:37.826626 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0107939-a8d4-46d4-9687-4c57b46d4d6f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:37.827108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0107939-a8d4-46d4-9687-4c57b46d4d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:37.827594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0107939-a8d4-46d4-9687-4c57b46d4d6f None admin] PATH_INFO: `/v3/users/ea013e6108e2453cbba79a5bd941750d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:37.829021 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0107939-a8d4-46d4-9687-4c57b46d4d6f None admin] RBAC: Authorizing `identity:get_user(user_id=ea013e6108e2453cbba79a5bd941750d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:37.830760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0107939-a8d4-46d4-9687-4c57b46d4d6f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:37.840431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 148/284] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:37 2022] GET /identity/v3/users/ea013e6108e2453cbba79a5bd941750d => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:37.845656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd3b1f-5602-4ed5-a265-33478187f7e9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:38.002190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd3b1f-5602-4ed5-a265-33478187f7e9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:38.026302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:38.052735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39ce76-ac0c-4f39-ae7a-d50673552227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:38.053827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39ce76-ac0c-4f39-ae7a-d50673552227 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:38.054141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39ce76-ac0c-4f39-ae7a-d50673552227 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:38.054419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39ce76-ac0c-4f39-ae7a-d50673552227 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:38.055644 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39ce76-ac0c-4f39-ae7a-d50673552227 None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:38.057082 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39ce76-ac0c-4f39-ae7a-d50673552227 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:38.059292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 137/285] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:37 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:38.064486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0107939-a8d4-46d4-9687-4c57b46d4d6f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:38.118917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ca648-cb68-4364-9c6f-aae879045910 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:38.119197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ca648-cb68-4364-9c6f-aae879045910 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:38.119476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ca648-cb68-4364-9c6f-aae879045910 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:38.121391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 138/286] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:38.126151 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e236e-ea07-4c36-9792-d2f183afaa9b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:38.126438 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e236e-ea07-4c36-9792-d2f183afaa9b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:38.126727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e236e-ea07-4c36-9792-d2f183afaa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:38.157837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-434e236e-ea07-4c36-9792-d2f183afaa9b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:38.159063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-434e236e-ea07-4c36-9792-d2f183afaa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:38.246590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 139/287] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:38.256762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:38.265525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0107939-a8d4-46d4-9687-4c57b46d4d6f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:38.291614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e236e-ea07-4c36-9792-d2f183afaa9b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:38.332014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ad6c4-3605-473a-8b97-ed0610204075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:38.333211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad6c4-3605-473a-8b97-ed0610204075 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:38.333638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad6c4-3605-473a-8b97-ed0610204075 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:38.334075 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad6c4-3605-473a-8b97-ed0610204075 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e/users/ea013e6108e2453cbba79a5bd941750d/roles/43bec89a20b54bdeb265c47e7af8d7b6` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:38.350976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ad6c4-3605-473a-8b97-ed0610204075 None admin] RBAC: Authorizing `identity:create_grant(project_id=35d52ce6b9e5491abc31216be989797e, user_id=ea013e6108e2453cbba79a5bd941750d, role_id=43bec89a20b54bdeb265c47e7af8d7b6)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:38.352683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ad6c4-3605-473a-8b97-ed0610204075 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:38.418537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e236e-ea07-4c36-9792-d2f183afaa9b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:38.429171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 149/288] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:47:38 2022] PUT /identity/v3/projects/35d52ce6b9e5491abc31216be989797e/users/ea013e6108e2453cbba79a5bd941750d/roles/43bec89a20b54bdeb265c47e7af8d7b6 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:38.443172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:38.467853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a498bf-3253-49e2-976b-73f9fd901384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:38.468951 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a498bf-3253-49e2-976b-73f9fd901384 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:38.469377 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a498bf-3253-49e2-976b-73f9fd901384 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:38.469794 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a498bf-3253-49e2-976b-73f9fd901384 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:38.482329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a498bf-3253-49e2-976b-73f9fd901384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b0a498bf-3253-49e2-976b-73f9fd901384 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:38.491024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:38.492140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:38.493139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:38.495801 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 140/289] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:38.501298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ad6c4-3605-473a-8b97-ed0610204075 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:38.679786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ad6c4-3605-473a-8b97-ed0610204075 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:38.720233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1db5c9-f75b-480b-8d69-da38ca194894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:38.721189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1db5c9-f75b-480b-8d69-da38ca194894 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:38.721628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1db5c9-f75b-480b-8d69-da38ca194894 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:38.722089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1db5c9-f75b-480b-8d69-da38ca194894 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:38.723034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1db5c9-f75b-480b-8d69-da38ca194894 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:38.723652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1db5c9-f75b-480b-8d69-da38ca194894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:38.725354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1db5c9-f75b-480b-8d69-da38ca194894 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:38.737683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 150/290] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:38.743058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a498bf-3253-49e2-976b-73f9fd901384 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:38.862974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a498bf-3253-49e2-976b-73f9fd901384 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:38.915657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f854229-9e1d-4ee5-88ec-bf06302e61e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:38.916779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f854229-9e1d-4ee5-88ec-bf06302e61e2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:38.917207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f854229-9e1d-4ee5-88ec-bf06302e61e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:38.917624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f854229-9e1d-4ee5-88ec-bf06302e61e2 None admin] PATH_INFO: `/v3/users/aedb73c4ce524c8091e33a9161a50b18` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:38.919006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f854229-9e1d-4ee5-88ec-bf06302e61e2 None admin] RBAC: Authorizing `identity:get_user(user_id=aedb73c4ce524c8091e33a9161a50b18)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:38.920675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f854229-9e1d-4ee5-88ec-bf06302e61e2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:38.922782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 141/291] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:38 2022] GET /identity/v3/users/aedb73c4ce524c8091e33a9161a50b18 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:38.928222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1db5c9-f75b-480b-8d69-da38ca194894 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:38.950345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:39.054389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1db5c9-f75b-480b-8d69-da38ca194894 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:39.101386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a41543-189e-4ee4-b4ab-d729e42b1223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:39.102120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a41543-189e-4ee4-b4ab-d729e42b1223 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:39.102370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a41543-189e-4ee4-b4ab-d729e42b1223 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:39.102653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a41543-189e-4ee4-b4ab-d729e42b1223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:39.193501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a41543-189e-4ee4-b4ab-d729e42b1223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:39.195026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a41543-189e-4ee4-b4ab-d729e42b1223 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:39.224750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:39.339044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 151/292] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:39.344519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f854229-9e1d-4ee5-88ec-bf06302e61e2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:39.425744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:39.501871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f854229-9e1d-4ee5-88ec-bf06302e61e2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:39.542008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d7ee8-99cb-47e3-b4e7-4df4114cc704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:39.542911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7ee8-99cb-47e3-b4e7-4df4114cc704 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:39.543194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7ee8-99cb-47e3-b4e7-4df4114cc704 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:39.543468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7ee8-99cb-47e3-b4e7-4df4114cc704 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:39.544652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d7ee8-99cb-47e3-b4e7-4df4114cc704 None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:39.546355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d7ee8-99cb-47e3-b4e7-4df4114cc704 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:39.548476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 142/293] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:39 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:39.554450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a41543-189e-4ee4-b4ab-d729e42b1223 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:39.716735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a41543-189e-4ee4-b4ab-d729e42b1223 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:39.742677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:39.767347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520acae-0743-4dee-be5d-0fc3ad3637a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:39.767926 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520acae-0743-4dee-be5d-0fc3ad3637a1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:39.768207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520acae-0743-4dee-be5d-0fc3ad3637a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:39.768483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520acae-0743-4dee-be5d-0fc3ad3637a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:39.769949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520acae-0743-4dee-be5d-0fc3ad3637a1 None admin] RBAC: Adding query filter params (role.id=a70b63ede7014fa9bbcb4e03275a2664, scope.project.id=35d52ce6b9e5491abc31216be989797e, user.id=aedb73c4ce524c8091e33a9161a50b18) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:39.770283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520acae-0743-4dee-be5d-0fc3ad3637a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:39.771739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520acae-0743-4dee-be5d-0fc3ad3637a1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:39.779968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5596fb-cdea-41de-b07e-e9d8dc4eb31d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:39.780252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5596fb-cdea-41de-b07e-e9d8dc4eb31d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:39.780519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 152/294] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:39 2022] GET /identity/v3/role_assignments?user.id=aedb73c4ce524c8091e33a9161a50b18&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:39.780623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5596fb-cdea-41de-b07e-e9d8dc4eb31d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:39.781986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 143/295] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:39.788212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffd60f-98a9-4e1f-beed-cb61056e002e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:39.788526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffd60f-98a9-4e1f-beed-cb61056e002e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:39.788759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffd60f-98a9-4e1f-beed-cb61056e002e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:39.815234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f5ffd60f-98a9-4e1f-beed-cb61056e002e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:39.816627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f5ffd60f-98a9-4e1f-beed-cb61056e002e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:39.908681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 153/296] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:39.958603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5596fb-cdea-41de-b07e-e9d8dc4eb31d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:40.097334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002dc0e-9bb2-4f7a-9e89-128b22ee3590 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.097334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002dc0e-9bb2-4f7a-9e89-128b22ee3590 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.097334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002dc0e-9bb2-4f7a-9e89-128b22ee3590 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.097960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5596fb-cdea-41de-b07e-e9d8dc4eb31d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:40.099671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 154/297] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:40.104145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724aa2-aaef-429f-9c95-467860eaf2b9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.104513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724aa2-aaef-429f-9c95-467860eaf2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.104867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724aa2-aaef-429f-9c95-467860eaf2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.108662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:40.126771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1e724aa2-aaef-429f-9c95-467860eaf2b9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:40.126975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1e724aa2-aaef-429f-9c95-467860eaf2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:40.138496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d58f2-de58-4d00-9890-141466b307d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:40.139586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d58f2-de58-4d00-9890-141466b307d4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.140152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d58f2-de58-4d00-9890-141466b307d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.140606 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d58f2-de58-4d00-9890-141466b307d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.153601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d58f2-de58-4d00-9890-141466b307d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-0e2d58f2-de58-4d00-9890-141466b307d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:40.163361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:40.164236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:40.165039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:40.168230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 144/298] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:40.174651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d58f2-de58-4d00-9890-141466b307d4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:40.215422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 155/299] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:40.216615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e754b2-5f5c-495c-b383-931449d94c88 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.216896 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e754b2-5f5c-495c-b383-931449d94c88 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.217171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e754b2-5f5c-495c-b383-931449d94c88 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.218459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 156/300] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:40.224210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f80609-c8f0-4c7f-9968-d2e51efd7d48 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.224496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f80609-c8f0-4c7f-9968-d2e51efd7d48 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.224773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f80609-c8f0-4c7f-9968-d2e51efd7d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.243787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-41f80609-c8f0-4c7f-9968-d2e51efd7d48 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:40.244976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-41f80609-c8f0-4c7f-9968-d2e51efd7d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:40.289389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:40.304242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d58f2-de58-4d00-9890-141466b307d4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:40.326498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 157/301] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:40.327370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f80609-c8f0-4c7f-9968-d2e51efd7d48 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:40.355714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e68ba-6018-46c9-ad46-a14540f0d251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:40.356713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e68ba-6018-46c9-ad46-a14540f0d251 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.357095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e68ba-6018-46c9-ad46-a14540f0d251 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.357377 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e68ba-6018-46c9-ad46-a14540f0d251 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.358162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e68ba-6018-46c9-ad46-a14540f0d251 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:40.358472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e68ba-6018-46c9-ad46-a14540f0d251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:40.359832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e68ba-6018-46c9-ad46-a14540f0d251 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:40.372846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 145/302] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:40.372846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e68ba-6018-46c9-ad46-a14540f0d251 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:40.403886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:40.449570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f80609-c8f0-4c7f-9968-d2e51efd7d48 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:40.477197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e68ba-6018-46c9-ad46-a14540f0d251 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:40.487943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5337cb-7fa3-4b21-b129-86c11f602608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:40.488738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5337cb-7fa3-4b21-b129-86c11f602608 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.489020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5337cb-7fa3-4b21-b129-86c11f602608 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.489302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5337cb-7fa3-4b21-b129-86c11f602608 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.490589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5337cb-7fa3-4b21-b129-86c11f602608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:40.492062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5337cb-7fa3-4b21-b129-86c11f602608 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:40.494075 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 158/303] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:40.494574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5337cb-7fa3-4b21-b129-86c11f602608 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:40.519417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6262a-55f0-4722-bbb4-d349203b9542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:40.520565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6262a-55f0-4722-bbb4-d349203b9542 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.520920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6262a-55f0-4722-bbb4-d349203b9542 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.521195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6262a-55f0-4722-bbb4-d349203b9542 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.531711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6262a-55f0-4722-bbb4-d349203b9542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:40.538861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-05d6262a-55f0-4722-bbb4-d349203b9542 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:40.541362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:40.542483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 146/304] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:40.551591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6262a-55f0-4722-bbb4-d349203b9542 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:40.603517 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5337cb-7fa3-4b21-b129-86c11f602608 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:40.638132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3b894-7be5-47d2-82d7-332212ab670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:40.639003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3b894-7be5-47d2-82d7-332212ab670f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.639282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3b894-7be5-47d2-82d7-332212ab670f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.639560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3b894-7be5-47d2-82d7-332212ab670f None admin] PATH_INFO: `/v3/users/57889db9516448b3871071927b1bfeae` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.645290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:40.653784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3b894-7be5-47d2-82d7-332212ab670f None admin] RBAC: Authorizing `identity:get_user(user_id=57889db9516448b3871071927b1bfeae)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:40.654108 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6262a-55f0-4722-bbb4-d349203b9542 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:40.655315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3b894-7be5-47d2-82d7-332212ab670f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:40.657017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 159/305] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:40 2022] GET /identity/v3/users/57889db9516448b3871071927b1bfeae => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:40.663442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3b894-7be5-47d2-82d7-332212ab670f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:40.693374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe7793-0b8f-42d4-922b-b6d75766da03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:40.694074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe7793-0b8f-42d4-922b-b6d75766da03 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.694352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe7793-0b8f-42d4-922b-b6d75766da03 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.694631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe7793-0b8f-42d4-922b-b6d75766da03 None admin] PATH_INFO: `/v3/groups` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.695315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe7793-0b8f-42d4-922b-b6d75766da03 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:40.696748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe7793-0b8f-42d4-922b-b6d75766da03 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:40.763312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:40.768053 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3b894-7be5-47d2-82d7-332212ab670f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:40.770123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 147/306] 10.222.0.40 () {60 vars in 1134 bytes} [Wed Jul 13 08:47:40 2022] POST /identity/v3/groups => generated 229 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:47:40.770807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe7793-0b8f-42d4-922b-b6d75766da03 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:40.818269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779e3b2-f7f8-4b68-9cca-41ed749fcdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:40.819374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779e3b2-f7f8-4b68-9cca-41ed749fcdba None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.820074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779e3b2-f7f8-4b68-9cca-41ed749fcdba None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.820591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779e3b2-f7f8-4b68-9cca-41ed749fcdba None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.821928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779e3b2-f7f8-4b68-9cca-41ed749fcdba None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:40.822508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779e3b2-f7f8-4b68-9cca-41ed749fcdba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:40.824160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779e3b2-f7f8-4b68-9cca-41ed749fcdba None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:40.835918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 160/307] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:40.841145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779e3b2-f7f8-4b68-9cca-41ed749fcdba None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:40.905484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe7793-0b8f-42d4-922b-b6d75766da03 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:40.950862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d41da1-4a85-44ae-9ed1-6c7191beb9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:40.951531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d41da1-4a85-44ae-9ed1-6c7191beb9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.951815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d41da1-4a85-44ae-9ed1-6c7191beb9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.952094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d41da1-4a85-44ae-9ed1-6c7191beb9e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.953663 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d41da1-4a85-44ae-9ed1-6c7191beb9e7 None admin] RBAC: Adding query filter params (role.id=43bec89a20b54bdeb265c47e7af8d7b6, user.id=57889db9516448b3871071927b1bfeae, scope.system=all) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:40.953984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d41da1-4a85-44ae-9ed1-6c7191beb9e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:40.955990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d41da1-4a85-44ae-9ed1-6c7191beb9e7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:40.963595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 148/308] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 13 08:47:40 2022] GET /identity/v3/role_assignments?user.id=57889db9516448b3871071927b1bfeae&scope.system=all&role.id=43bec89a20b54bdeb265c47e7af8d7b6 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:40.974267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b8575-ccc1-4763-9050-4cd7b6fedee6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.974575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b8575-ccc1-4763-9050-4cd7b6fedee6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.975242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b8575-ccc1-4763-9050-4cd7b6fedee6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:40.977395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 149/309] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:47:40.981891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779e3b2-f7f8-4b68-9cca-41ed749fcdba None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:40.982670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14930ba8-d107-44c4-8700-c8c4a84ed729 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:40.983021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14930ba8-d107-44c4-8700-c8c4a84ed729 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:40.983305 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14930ba8-d107-44c4-8700-c8c4a84ed729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:41.004524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-14930ba8-d107-44c4-8700-c8c4a84ed729 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:41.005801 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-14930ba8-d107-44c4-8700-c8c4a84ed729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:41.020943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:41.040966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db276d15-dda6-40d3-b3ed-2e217aeb897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:41.041958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db276d15-dda6-40d3-b3ed-2e217aeb897c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:41.042312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db276d15-dda6-40d3-b3ed-2e217aeb897c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:41.042967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db276d15-dda6-40d3-b3ed-2e217aeb897c None admin] PATH_INFO: `/v3/users/b7f15ae95c434a9d87f0ef44ecb3f009` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:41.044062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db276d15-dda6-40d3-b3ed-2e217aeb897c None admin] RBAC: Authorizing `identity:get_user(user_id=b7f15ae95c434a9d87f0ef44ecb3f009)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:41.045613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db276d15-dda6-40d3-b3ed-2e217aeb897c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:41.048276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 161/310] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:40 2022] GET /identity/v3/users/b7f15ae95c434a9d87f0ef44ecb3f009 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:41.056015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db276d15-dda6-40d3-b3ed-2e217aeb897c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:41.092326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:41.093827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 150/311] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:41.117395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14930ba8-d107-44c4-8700-c8c4a84ed729 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:41.179458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db276d15-dda6-40d3-b3ed-2e217aeb897c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:41.221497 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14930ba8-d107-44c4-8700-c8c4a84ed729 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:41.225159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0106b02-0846-4e73-929b-2660d9c920b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:41.226922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0106b02-0846-4e73-929b-2660d9c920b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:41.227249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0106b02-0846-4e73-929b-2660d9c920b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:41.227666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0106b02-0846-4e73-929b-2660d9c920b7 None admin] PATH_INFO: `/v3/system/users/b7f15ae95c434a9d87f0ef44ecb3f009/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:41.229855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0106b02-0846-4e73-929b-2660d9c920b7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b7f15ae95c434a9d87f0ef44ecb3f009, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:41.232826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0106b02-0846-4e73-929b-2660d9c920b7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:41.259073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa16a0-8128-45f4-a4d8-bfc611298f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:41.259965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa16a0-8128-45f4-a4d8-bfc611298f23 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:41.260315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa16a0-8128-45f4-a4d8-bfc611298f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:41.260654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa16a0-8128-45f4-a4d8-bfc611298f23 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:41.272930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa16a0-8128-45f4-a4d8-bfc611298f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7daa16a0-8128-45f4-a4d8-bfc611298f23 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:41.280938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:41.281798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:41.282640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:41.287513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 151/312] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:41.288318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa16a0-8128-45f4-a4d8-bfc611298f23 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:41.299323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:41.341362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 162/313] 10.222.0.40 () {58 vars in 1334 bytes} [Wed Jul 13 08:47:41 2022] PUT /identity/v3/system/users/b7f15ae95c434a9d87f0ef44ecb3f009/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:41.407082 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa16a0-8128-45f4-a4d8-bfc611298f23 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:41.446713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad507e50-5d13-4218-a8dc-872ac84a2683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:41.447304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad507e50-5d13-4218-a8dc-872ac84a2683 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:41.447581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad507e50-5d13-4218-a8dc-872ac84a2683 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:41.447848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad507e50-5d13-4218-a8dc-872ac84a2683 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:41.448597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad507e50-5d13-4218-a8dc-872ac84a2683 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:41.448898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad507e50-5d13-4218-a8dc-872ac84a2683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:41.450408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad507e50-5d13-4218-a8dc-872ac84a2683 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:41.460587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 152/314] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:41.466540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0106b02-0846-4e73-929b-2660d9c920b7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:41.568441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:41.577015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0106b02-0846-4e73-929b-2660d9c920b7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:41.612385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accb316-4b3c-4447-9877-81520d172928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:41.613576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accb316-4b3c-4447-9877-81520d172928 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:41.613861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accb316-4b3c-4447-9877-81520d172928 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:41.614166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accb316-4b3c-4447-9877-81520d172928 None admin] PATH_INFO: `/v3/users/ea013e6108e2453cbba79a5bd941750d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:41.615384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accb316-4b3c-4447-9877-81520d172928 None admin] RBAC: Authorizing `identity:get_user(user_id=ea013e6108e2453cbba79a5bd941750d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:41.616936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accb316-4b3c-4447-9877-81520d172928 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:41.618594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 163/315] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:41 2022] GET /identity/v3/users/ea013e6108e2453cbba79a5bd941750d => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:41.624136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad507e50-5d13-4218-a8dc-872ac84a2683 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:41.698933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:41.733281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad507e50-5d13-4218-a8dc-872ac84a2683 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:41.770919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd5977-0d0a-4a79-a7b1-837e45caf929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:41.771571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd5977-0d0a-4a79-a7b1-837e45caf929 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:41.771855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd5977-0d0a-4a79-a7b1-837e45caf929 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:41.772135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd5977-0d0a-4a79-a7b1-837e45caf929 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:41.842868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd5977-0d0a-4a79-a7b1-837e45caf929 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:41.844341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd5977-0d0a-4a79-a7b1-837e45caf929 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:41.911260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:41.946926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 153/316] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:41.952789 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accb316-4b3c-4447-9877-81520d172928 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:42.069872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accb316-4b3c-4447-9877-81520d172928 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:42.106141 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905071d3-46e8-43e6-9c6b-859ae0f64e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:42.107384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905071d3-46e8-43e6-9c6b-859ae0f64e0a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:42.107885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905071d3-46e8-43e6-9c6b-859ae0f64e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:42.108367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905071d3-46e8-43e6-9c6b-859ae0f64e0a None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:42.109937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905071d3-46e8-43e6-9c6b-859ae0f64e0a None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:42.111950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905071d3-46e8-43e6-9c6b-859ae0f64e0a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:42.114947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 164/317] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:41 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:42.121104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd5977-0d0a-4a79-a7b1-837e45caf929 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:42.227045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd5977-0d0a-4a79-a7b1-837e45caf929 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:42.262133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af1fa8-0242-422a-903b-ef254849995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:42.262785 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1fa8-0242-422a-903b-ef254849995e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:42.263066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1fa8-0242-422a-903b-ef254849995e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:42.263341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af1fa8-0242-422a-903b-ef254849995e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:42.264160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af1fa8-0242-422a-903b-ef254849995e None admin] RBAC: Adding query filter params (role.id=43bec89a20b54bdeb265c47e7af8d7b6, scope.project.id=35d52ce6b9e5491abc31216be989797e, user.id=ea013e6108e2453cbba79a5bd941750d) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:42.264472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af1fa8-0242-422a-903b-ef254849995e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:42.265901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af1fa8-0242-422a-903b-ef254849995e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:42.272718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 154/318] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:47:42 2022] GET /identity/v3/role_assignments?user.id=ea013e6108e2453cbba79a5bd941750d&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=43bec89a20b54bdeb265c47e7af8d7b6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:43.011789 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe5ca5-548a-42ef-80bc-bf691e89ac8e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.012308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe5ca5-548a-42ef-80bc-bf691e89ac8e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.012383 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe5ca5-548a-42ef-80bc-bf691e89ac8e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.013799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 165/319] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:43.019049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011cb83-1451-4479-905d-1f841acc68d1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.019342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011cb83-1451-4479-905d-1f841acc68d1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.019620 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011cb83-1451-4479-905d-1f841acc68d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.036974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7011cb83-1451-4479-905d-1f841acc68d1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:43.038278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7011cb83-1451-4479-905d-1f841acc68d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:43.042238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bcf07-bdad-41b3-b69f-726be1bdb5b3 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.042532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bcf07-bdad-41b3-b69f-726be1bdb5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.042810 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bcf07-bdad-41b3-b69f-726be1bdb5b3 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.044159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 166/320] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:43.049511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3d45f-668b-48e3-bc3c-c87d1c2758ca None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.058377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3d45f-668b-48e3-bc3c-c87d1c2758ca None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.058377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3d45f-668b-48e3-bc3c-c87d1c2758ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.067931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b6c3d45f-668b-48e3-bc3c-c87d1c2758ca None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:43.068370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:43.069182 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b6c3d45f-668b-48e3-bc3c-c87d1c2758ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:43.093862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:43.117246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 155/321] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:43.140218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011cb83-1451-4479-905d-1f841acc68d1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:43.140938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 167/322] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:43.163612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3d45f-668b-48e3-bc3c-c87d1c2758ca None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:43.246073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011cb83-1451-4479-905d-1f841acc68d1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:43.279651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3d45f-668b-48e3-bc3c-c87d1c2758ca None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:43.284098 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6f8e7-20e5-456f-80f1-05cf6a545682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:43.285138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6f8e7-20e5-456f-80f1-05cf6a545682 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.285628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6f8e7-20e5-456f-80f1-05cf6a545682 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.286124 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6f8e7-20e5-456f-80f1-05cf6a545682 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.287599 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6f8e7-20e5-456f-80f1-05cf6a545682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:43.289363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6f8e7-20e5-456f-80f1-05cf6a545682 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:43.291319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 156/323] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:47:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:43.296852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6f8e7-20e5-456f-80f1-05cf6a545682 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:43.317721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1e7ad-ece2-4afb-a5da-0cfe348027a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:43.318570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1e7ad-ece2-4afb-a5da-0cfe348027a3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.318848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1e7ad-ece2-4afb-a5da-0cfe348027a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.319116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1e7ad-ece2-4afb-a5da-0cfe348027a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.327109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:43.329641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1e7ad-ece2-4afb-a5da-0cfe348027a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8ee1e7ad-ece2-4afb-a5da-0cfe348027a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:43.338804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:43.345953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 168/324] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:43.347436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1e7ad-ece2-4afb-a5da-0cfe348027a3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:43.370959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:43.411050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6f8e7-20e5-456f-80f1-05cf6a545682 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:43.449131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba705f2-eafc-4927-872c-3604eec8cd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:43.449799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba705f2-eafc-4927-872c-3604eec8cd47 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.450103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba705f2-eafc-4927-872c-3604eec8cd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.450379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba705f2-eafc-4927-872c-3604eec8cd47 None admin] PATH_INFO: `/v3/groups` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.451067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba705f2-eafc-4927-872c-3604eec8cd47 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:43.452502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba705f2-eafc-4927-872c-3604eec8cd47 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:43.462432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1e7ad-ece2-4afb-a5da-0cfe348027a3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:43.497899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22b0fc-a639-484e-b3f9-b3e8ab5d1213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:43.498651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22b0fc-a639-484e-b3f9-b3e8ab5d1213 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.498931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22b0fc-a639-484e-b3f9-b3e8ab5d1213 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.499214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22b0fc-a639-484e-b3f9-b3e8ab5d1213 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.499958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22b0fc-a639-484e-b3f9-b3e8ab5d1213 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:43.500265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22b0fc-a639-484e-b3f9-b3e8ab5d1213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:43.501639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22b0fc-a639-484e-b3f9-b3e8ab5d1213 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:43.510690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 169/325] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:43.511892 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ccf3c-791c-4ab7-9b16-27d615b4fb1d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.512176 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ccf3c-791c-4ab7-9b16-27d615b4fb1d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.512458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ccf3c-791c-4ab7-9b16-27d615b4fb1d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.513721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 170/326] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:43.516192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ccf3c-791c-4ab7-9b16-27d615b4fb1d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:43.548817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 157/327] 10.222.0.40 () {60 vars in 1134 bytes} [Wed Jul 13 08:47:43 2022] POST /identity/v3/groups => generated 226 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:47:43.550552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4d425-bead-43fb-a9cc-a7363c190985 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.550864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4d425-bead-43fb-a9cc-a7363c190985 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.551141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4d425-bead-43fb-a9cc-a7363c190985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.570629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-04f4d425-bead-43fb-a9cc-a7363c190985 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:43.571882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-04f4d425-bead-43fb-a9cc-a7363c190985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:43.612226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:43.636403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ccf3c-791c-4ab7-9b16-27d615b4fb1d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:43.648515 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 158/328] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:43.672586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4d425-bead-43fb-a9cc-a7363c190985 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:43.674544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdf2fe-6d5d-4b36-8e86-279a6e590d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:43.675998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdf2fe-6d5d-4b36-8e86-279a6e590d9a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.677090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdf2fe-6d5d-4b36-8e86-279a6e590d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.677090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdf2fe-6d5d-4b36-8e86-279a6e590d9a None admin] PATH_INFO: `/v3/users/57889db9516448b3871071927b1bfeae` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.677753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdf2fe-6d5d-4b36-8e86-279a6e590d9a None admin] RBAC: Authorizing `identity:get_user(user_id=57889db9516448b3871071927b1bfeae)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:43.679262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdf2fe-6d5d-4b36-8e86-279a6e590d9a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:43.681152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 171/329] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:43 2022] GET /identity/v3/users/57889db9516448b3871071927b1bfeae => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:43.686646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdf2fe-6d5d-4b36-8e86-279a6e590d9a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:43.698177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:43.803455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdf2fe-6d5d-4b36-8e86-279a6e590d9a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:43.804226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4d425-bead-43fb-a9cc-a7363c190985 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:43.850830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae23db-dc23-469b-ba39-6ff63d8faaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:43.851326 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db51b7-e17f-41f8-86f9-000d6fbdec26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:43.851705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae23db-dc23-469b-ba39-6ff63d8faaaf None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.851988 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae23db-dc23-469b-ba39-6ff63d8faaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.852161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db51b7-e17f-41f8-86f9-000d6fbdec26 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:43.852253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae23db-dc23-469b-ba39-6ff63d8faaaf None admin] PATH_INFO: `/v3/system/users/57889db9516448b3871071927b1bfeae/roles/43bec89a20b54bdeb265c47e7af8d7b6` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.852401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db51b7-e17f-41f8-86f9-000d6fbdec26 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:43.852683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db51b7-e17f-41f8-86f9-000d6fbdec26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:43.853787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae23db-dc23-469b-ba39-6ff63d8faaaf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=57889db9516448b3871071927b1bfeae, role_id=43bec89a20b54bdeb265c47e7af8d7b6)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:43.855336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae23db-dc23-469b-ba39-6ff63d8faaaf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:43.864439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db51b7-e17f-41f8-86f9-000d6fbdec26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-42db51b7-e17f-41f8-86f9-000d6fbdec26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:43.872325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:43.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:43.874520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 159/330] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:43.880513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db51b7-e17f-41f8-86f9-000d6fbdec26 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:43.926570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 172/331] 10.222.0.40 () {58 vars in 1334 bytes} [Wed Jul 13 08:47:43 2022] PUT /identity/v3/system/users/57889db9516448b3871071927b1bfeae/roles/43bec89a20b54bdeb265c47e7af8d7b6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 08:47:43.983651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:44.008571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db51b7-e17f-41f8-86f9-000d6fbdec26 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:44.049272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271301b6-d7b1-4cde-9ba1-83b66d6dd448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:44.050503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271301b6-d7b1-4cde-9ba1-83b66d6dd448 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:44.050797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271301b6-d7b1-4cde-9ba1-83b66d6dd448 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:44.051062 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271301b6-d7b1-4cde-9ba1-83b66d6dd448 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:44.051833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271301b6-d7b1-4cde-9ba1-83b66d6dd448 None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:44.052143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271301b6-d7b1-4cde-9ba1-83b66d6dd448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:44.053678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271301b6-d7b1-4cde-9ba1-83b66d6dd448 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:44.064057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 160/332] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:44.070401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae23db-dc23-469b-ba39-6ff63d8faaaf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:44.088494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:44.182427 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae23db-dc23-469b-ba39-6ff63d8faaaf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:44.223818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5210c34-cec4-489d-a39f-41a4c3789a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:44.224696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5210c34-cec4-489d-a39f-41a4c3789a9d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:44.224978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5210c34-cec4-489d-a39f-41a4c3789a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:44.225254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5210c34-cec4-489d-a39f-41a4c3789a9d None admin] PATH_INFO: `/v3/users/b7f15ae95c434a9d87f0ef44ecb3f009` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:44.226494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5210c34-cec4-489d-a39f-41a4c3789a9d None admin] RBAC: Authorizing `identity:get_user(user_id=b7f15ae95c434a9d87f0ef44ecb3f009)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:44.227957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5210c34-cec4-489d-a39f-41a4c3789a9d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:44.229938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 173/333] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:44 2022] GET /identity/v3/users/b7f15ae95c434a9d87f0ef44ecb3f009 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:44.235386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271301b6-d7b1-4cde-9ba1-83b66d6dd448 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:44.355697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271301b6-d7b1-4cde-9ba1-83b66d6dd448 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:44.400851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa6b81-7de8-4f9f-8773-2b2fcd60fe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:44.401865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa6b81-7de8-4f9f-8773-2b2fcd60fe12 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:44.402404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa6b81-7de8-4f9f-8773-2b2fcd60fe12 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:44.402870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa6b81-7de8-4f9f-8773-2b2fcd60fe12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:44.403878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa6b81-7de8-4f9f-8773-2b2fcd60fe12 None admin] RBAC: Adding query filter params (role.id=a70b63ede7014fa9bbcb4e03275a2664, user.id=b7f15ae95c434a9d87f0ef44ecb3f009, scope.system=all) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:44.404378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa6b81-7de8-4f9f-8773-2b2fcd60fe12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:44.406018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa6b81-7de8-4f9f-8773-2b2fcd60fe12 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:44.417918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:44.422205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 161/334] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 13 08:47:44 2022] GET /identity/v3/role_assignments?user.id=b7f15ae95c434a9d87f0ef44ecb3f009&scope.system=all&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:46.324969 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e56413-a53c-4b64-9e44-2d75171628c0 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:46.359173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e56413-a53c-4b64-9e44-2d75171628c0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:46.359173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e56413-a53c-4b64-9e44-2d75171628c0 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:46.359173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 174/335] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:46.359959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443abcef-923f-4871-ab2c-a4d8f5d99693 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:46.359959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443abcef-923f-4871-ab2c-a4d8f5d99693 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:46.359959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443abcef-923f-4871-ab2c-a4d8f5d99693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:46.361574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-443abcef-923f-4871-ab2c-a4d8f5d99693 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:46.363360 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-443abcef-923f-4871-ab2c-a4d8f5d99693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:46.458241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 162/336] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:46.466973 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e373de2-4bbc-4041-a4b7-3b371b471aee None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:46.467550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e373de2-4bbc-4041-a4b7-3b371b471aee None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:46.468030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e373de2-4bbc-4041-a4b7-3b371b471aee None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:46.469579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 175/337] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:46.474531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3201970-ba0a-42c5-8a45-0e24ad17c1d2 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:46.474825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3201970-ba0a-42c5-8a45-0e24ad17c1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:46.475104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3201970-ba0a-42c5-8a45-0e24ad17c1d2 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:46.476711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 163/338] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:46.482200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40c8cf-aac9-4f8f-aa17-0be0b06e3269 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:46.482736 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40c8cf-aac9-4f8f-aa17-0be0b06e3269 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:46.483014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587e338-6b81-4508-87b8-cd87ead3363a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:46.483313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587e338-6b81-4508-87b8-cd87ead3363a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:46.483598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587e338-6b81-4508-87b8-cd87ead3363a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:46.483869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40c8cf-aac9-4f8f-aa17-0be0b06e3269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:46.506586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8d40c8cf-aac9-4f8f-aa17-0be0b06e3269 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:46.508126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8d40c8cf-aac9-4f8f-aa17-0be0b06e3269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:46.509485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1587e338-6b81-4508-87b8-cd87ead3363a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:46.510696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1587e338-6b81-4508-87b8-cd87ead3363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:46.583640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 164/339] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:46.584562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587e338-6b81-4508-87b8-cd87ead3363a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:46.589321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 176/340] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:46.615365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40c8cf-aac9-4f8f-aa17-0be0b06e3269 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:46.630773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:46.673585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:46.720537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587e338-6b81-4508-87b8-cd87ead3363a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:46.749877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40c8cf-aac9-4f8f-aa17-0be0b06e3269 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:46.763079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072df4f1-261b-4796-93f2-1de59473dc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:46.763900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072df4f1-261b-4796-93f2-1de59473dc22 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:46.764186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072df4f1-261b-4796-93f2-1de59473dc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:46.764460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072df4f1-261b-4796-93f2-1de59473dc22 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:46.778774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072df4f1-261b-4796-93f2-1de59473dc22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:46.787449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f98de-573d-48aa-961c-4384c80cd972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-072df4f1-261b-4796-93f2-1de59473dc22 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:46.788050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:46.788940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:46.790258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:46.791177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f98de-573d-48aa-961c-4384c80cd972 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:46.791177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f98de-573d-48aa-961c-4384c80cd972 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:46.791177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f98de-573d-48aa-961c-4384c80cd972 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:46.791371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 165/341] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:46.791371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072df4f1-261b-4796-93f2-1de59473dc22 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:46.805811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f98de-573d-48aa-961c-4384c80cd972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a04f98de-573d-48aa-961c-4384c80cd972 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:46.821919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:46.822885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 177/342] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:46.824455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f98de-573d-48aa-961c-4384c80cd972 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:46.908048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072df4f1-261b-4796-93f2-1de59473dc22 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:46.948375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e35a2d-e3b7-4c78-b5cf-01066879c851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:46.949453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e35a2d-e3b7-4c78-b5cf-01066879c851 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:46.949743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e35a2d-e3b7-4c78-b5cf-01066879c851 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:46.950025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e35a2d-e3b7-4c78-b5cf-01066879c851 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:46.950881 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:46.964339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e35a2d-e3b7-4c78-b5cf-01066879c851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:46.966692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-e0e35a2d-e3b7-4c78-b5cf-01066879c851 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:46.975297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:46.976218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:46.977454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 166/343] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:46.978161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e35a2d-e3b7-4c78-b5cf-01066879c851 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:47.011518 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f98de-573d-48aa-961c-4384c80cd972 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:47.060455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d16a46-c2e7-41a2-b7c7-5184da636954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:47.061142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d16a46-c2e7-41a2-b7c7-5184da636954 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:47.061429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d16a46-c2e7-41a2-b7c7-5184da636954 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:47.061710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d16a46-c2e7-41a2-b7c7-5184da636954 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:47.062522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d16a46-c2e7-41a2-b7c7-5184da636954 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:47.062833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d16a46-c2e7-41a2-b7c7-5184da636954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:47.064231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d16a46-c2e7-41a2-b7c7-5184da636954 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:47.080316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 178/344] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:47.081799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d16a46-c2e7-41a2-b7c7-5184da636954 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:47.107460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e35a2d-e3b7-4c78-b5cf-01066879c851 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:47.142547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a74d5-ab60-4431-a0b5-b817a020da57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:47.143211 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a74d5-ab60-4431-a0b5-b817a020da57 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:47.143492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a74d5-ab60-4431-a0b5-b817a020da57 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:47.144281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a74d5-ab60-4431-a0b5-b817a020da57 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:47.145316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a74d5-ab60-4431-a0b5-b817a020da57 None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:47.145621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a74d5-ab60-4431-a0b5-b817a020da57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:47.147024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a74d5-ab60-4431-a0b5-b817a020da57 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:47.156004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 167/345] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:47.157308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8492b6c-8526-4918-a81f-92eca2015375 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:47.157612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8492b6c-8526-4918-a81f-92eca2015375 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:47.157883 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8492b6c-8526-4918-a81f-92eca2015375 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:47.159263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 168/346] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:47.160083 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8492b6c-8526-4918-a81f-92eca2015375 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:47.200010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d16a46-c2e7-41a2-b7c7-5184da636954 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:47.212054 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:47.234436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:47.241353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dabfe2-b132-468a-9085-c7dcfee23229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:47.242071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dabfe2-b132-468a-9085-c7dcfee23229 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:47.242350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dabfe2-b132-468a-9085-c7dcfee23229 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:47.242631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dabfe2-b132-468a-9085-c7dcfee23229 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:47.243661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dabfe2-b132-468a-9085-c7dcfee23229 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:47.243970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dabfe2-b132-468a-9085-c7dcfee23229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:47.245404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dabfe2-b132-468a-9085-c7dcfee23229 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:47.254902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 179/347] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:47.255737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dabfe2-b132-468a-9085-c7dcfee23229 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:47.272933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8492b6c-8526-4918-a81f-92eca2015375 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:47.309357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b227f-445e-4c26-897c-c1329b021332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:47.310013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b227f-445e-4c26-897c-c1329b021332 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:47.310337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b227f-445e-4c26-897c-c1329b021332 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:47.310610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b227f-445e-4c26-897c-c1329b021332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:47.375945 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dabfe2-b132-468a-9085-c7dcfee23229 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:47.390633 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b227f-445e-4c26-897c-c1329b021332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:47.392089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b227f-445e-4c26-897c-c1329b021332 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:47.418967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc833bd-e6c8-4127-8e52-43328ff6c749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:47.419552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc833bd-e6c8-4127-8e52-43328ff6c749 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:47.419833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc833bd-e6c8-4127-8e52-43328ff6c749 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:47.420206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc833bd-e6c8-4127-8e52-43328ff6c749 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:47.436163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:47.494183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:47.500660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 169/348] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:47.501927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1018d-e447-48d0-918f-d97747c1beaf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:47.502237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1018d-e447-48d0-918f-d97747c1beaf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:47.502507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1018d-e447-48d0-918f-d97747c1beaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:47.508031 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc833bd-e6c8-4127-8e52-43328ff6c749 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:47.509621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc833bd-e6c8-4127-8e52-43328ff6c749 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:47.520420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-20c1018d-e447-48d0-918f-d97747c1beaf None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:47.521600 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-20c1018d-e447-48d0-918f-d97747c1beaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:47.605018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 170/349] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:47.605974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1018d-e447-48d0-918f-d97747c1beaf None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:47.658376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 180/350] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:47.659798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc833bd-e6c8-4127-8e52-43328ff6c749 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:47.701500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:47.742611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1018d-e447-48d0-918f-d97747c1beaf None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:47.773216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:47.781410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc833bd-e6c8-4127-8e52-43328ff6c749 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:47.794587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c246e5-eaf7-4ed8-880f-adf53c34a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:47.795752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c246e5-eaf7-4ed8-880f-adf53c34a308 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:47.796289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c246e5-eaf7-4ed8-880f-adf53c34a308 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:47.796564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c246e5-eaf7-4ed8-880f-adf53c34a308 None admin] PATH_INFO: `/v3/users/57889db9516448b3871071927b1bfeae` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:47.797928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c246e5-eaf7-4ed8-880f-adf53c34a308 None admin] RBAC: Authorizing `identity:get_user(user_id=57889db9516448b3871071927b1bfeae)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:47.800471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c246e5-eaf7-4ed8-880f-adf53c34a308 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:47.803185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 171/351] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:47:47 2022] GET /identity/v3/users/57889db9516448b3871071927b1bfeae => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:47.803972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c246e5-eaf7-4ed8-880f-adf53c34a308 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:47.830449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12442f98-eae9-4550-ba5f-ab6d5d9760ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:47.832104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12442f98-eae9-4550-ba5f-ab6d5d9760ad None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:47.832477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12442f98-eae9-4550-ba5f-ab6d5d9760ad None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:47.833052 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12442f98-eae9-4550-ba5f-ab6d5d9760ad None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:47.834297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12442f98-eae9-4550-ba5f-ab6d5d9760ad None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:47.835746 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12442f98-eae9-4550-ba5f-ab6d5d9760ad None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:47.842115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 181/352] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:47 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:47.842965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12442f98-eae9-4550-ba5f-ab6d5d9760ad None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:47.945086 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c246e5-eaf7-4ed8-880f-adf53c34a308 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:47.968976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12442f98-eae9-4550-ba5f-ab6d5d9760ad None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:47.990086 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aa76e-a292-40ff-9597-783ac9287c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:47.990450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aa76e-a292-40ff-9597-783ac9287c7b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:47.990791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aa76e-a292-40ff-9597-783ac9287c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:47.991166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aa76e-a292-40ff-9597-783ac9287c7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.008595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:48.012841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0aa76e-a292-40ff-9597-783ac9287c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:48.022147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e249d-2a5a-467e-9dd8-e185cc0d058f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-6a0aa76e-a292-40ff-9597-783ac9287c7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:48.022466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:48.026936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:48.027745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:48.028599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e249d-2a5a-467e-9dd8-e185cc0d058f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:48.028599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e249d-2a5a-467e-9dd8-e185cc0d058f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:48.028599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e249d-2a5a-467e-9dd8-e185cc0d058f None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.028797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 172/353] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:48.028797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aa76e-a292-40ff-9597-783ac9287c7b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:48.028924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e249d-2a5a-467e-9dd8-e185cc0d058f None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:48.028924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e249d-2a5a-467e-9dd8-e185cc0d058f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:48.028924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 182/354] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:47 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:48.028924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e249d-2a5a-467e-9dd8-e185cc0d058f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:48.073243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:48.164792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aa76e-a292-40ff-9597-783ac9287c7b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:48.169248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e249d-2a5a-467e-9dd8-e185cc0d058f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:48.213840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfb56b-b502-4f8f-84da-6e5d12e131aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:48.214426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40f750-7dd0-4d9d-8e86-e638babdacc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:48.214685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfb56b-b502-4f8f-84da-6e5d12e131aa None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:48.215031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfb56b-b502-4f8f-84da-6e5d12e131aa None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:48.215284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40f750-7dd0-4d9d-8e86-e638babdacc3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:48.215380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfb56b-b502-4f8f-84da-6e5d12e131aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.215571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40f750-7dd0-4d9d-8e86-e638babdacc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:48.216210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40f750-7dd0-4d9d-8e86-e638babdacc3 None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.216305 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfb56b-b502-4f8f-84da-6e5d12e131aa None admin] RBAC: Adding query filter params (role.id=43bec89a20b54bdeb265c47e7af8d7b6, user.id=57889db9516448b3871071927b1bfeae, scope.system=all) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:48.216687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfb56b-b502-4f8f-84da-6e5d12e131aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:48.220818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfb56b-b502-4f8f-84da-6e5d12e131aa None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:48.223369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40f750-7dd0-4d9d-8e86-e638babdacc3 None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:48.224829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40f750-7dd0-4d9d-8e86-e638babdacc3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:48.227740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 183/355] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:48 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:48.228984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0127db-3af5-4e98-bc59-6fed6446f58d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:48.229280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0127db-3af5-4e98-bc59-6fed6446f58d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:48.229560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0127db-3af5-4e98-bc59-6fed6446f58d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.230985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 184/356] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:48.231644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0127db-3af5-4e98-bc59-6fed6446f58d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:48.235074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 173/357] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 13 08:47:48 2022] GET /identity/v3/role_assignments?user.id=57889db9516448b3871071927b1bfeae&scope.system=all&role.id=43bec89a20b54bdeb265c47e7af8d7b6 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:48.235802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfb56b-b502-4f8f-84da-6e5d12e131aa None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:48.331469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:48.366440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:48.374117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0127db-3af5-4e98-bc59-6fed6446f58d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:48.399104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfb56b-b502-4f8f-84da-6e5d12e131aa None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:48.422463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48c90f-f9c9-49b1-892f-e4d1efae5f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:48.423133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48c90f-f9c9-49b1-892f-e4d1efae5f89 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:48.423418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48c90f-f9c9-49b1-892f-e4d1efae5f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:48.423693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48c90f-f9c9-49b1-892f-e4d1efae5f89 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.424555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48c90f-f9c9-49b1-892f-e4d1efae5f89 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:48.424901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48c90f-f9c9-49b1-892f-e4d1efae5f89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:48.426349 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48c90f-f9c9-49b1-892f-e4d1efae5f89 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:48.436588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 185/358] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:48.438524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e41a2-5648-4500-84a9-f4f77cb58400 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:48.438814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e41a2-5648-4500-84a9-f4f77cb58400 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:48.439087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e41a2-5648-4500-84a9-f4f77cb58400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.447664 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94a8f8-ab3a-445f-abb3-93bf432ff8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:48.448971 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94a8f8-ab3a-445f-abb3-93bf432ff8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:48.449464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94a8f8-ab3a-445f-abb3-93bf432ff8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:48.449939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94a8f8-ab3a-445f-abb3-93bf432ff8e1 None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.452262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94a8f8-ab3a-445f-abb3-93bf432ff8e1 None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:48.454681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94a8f8-ab3a-445f-abb3-93bf432ff8e1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:48.457588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 174/359] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:48 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:48.458616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94a8f8-ab3a-445f-abb3-93bf432ff8e1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:48.459767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8e6e41a2-5648-4500-84a9-f4f77cb58400 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:48.460560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8e6e41a2-5648-4500-84a9-f4f77cb58400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:48.567981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 186/360] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:48.568538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e41a2-5648-4500-84a9-f4f77cb58400 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:48.599697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94a8f8-ab3a-445f-abb3-93bf432ff8e1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:48.629149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:48.638563 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb68847-f589-4f4c-9dea-980b1750461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:48.639224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68847-f589-4f4c-9dea-980b1750461c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:48.639511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68847-f589-4f4c-9dea-980b1750461c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:48.639784 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68847-f589-4f4c-9dea-980b1750461c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.641073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb68847-f589-4f4c-9dea-980b1750461c None admin] RBAC: Adding query filter params (group.id=b0fd84213ca0455aaf2983a2b994abb9, role.id=120eeffad7094e10a69166d19aaa6439, scope.project.id=245d48325d884a83b0a57436df3e1418) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:48.641073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb68847-f589-4f4c-9dea-980b1750461c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:48.642363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb68847-f589-4f4c-9dea-980b1750461c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:48.648952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 175/361] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 13 08:47:48 2022] GET /identity/v3/role_assignments?group.id=b0fd84213ca0455aaf2983a2b994abb9&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=120eeffad7094e10a69166d19aaa6439 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:48.653907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb68847-f589-4f4c-9dea-980b1750461c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:48.692938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:48.713107 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e41a2-5648-4500-84a9-f4f77cb58400 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:48.763378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc6917-baf5-4e2a-9772-49f4019e45df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:48.764050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc6917-baf5-4e2a-9772-49f4019e45df None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:48.764830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc6917-baf5-4e2a-9772-49f4019e45df None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:48.765116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc6917-baf5-4e2a-9772-49f4019e45df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.781215 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb68847-f589-4f4c-9dea-980b1750461c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:48.838126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ef33-5ac1-429f-87bc-92cbabdfedc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:48.838126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ef33-5ac1-429f-87bc-92cbabdfedc1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:48.838126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ef33-5ac1-429f-87bc-92cbabdfedc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:48.838126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ef33-5ac1-429f-87bc-92cbabdfedc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:48.838499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ef33-5ac1-429f-87bc-92cbabdfedc1 None admin] RBAC: Adding query filter params (group.id=b0fd84213ca0455aaf2983a2b994abb9, role.id=a70b63ede7014fa9bbcb4e03275a2664, scope.project.id=245d48325d884a83b0a57436df3e1418) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:48.838499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ef33-5ac1-429f-87bc-92cbabdfedc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:48.839708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ef33-5ac1-429f-87bc-92cbabdfedc1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:48.847504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 176/362] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 13 08:47:48 2022] GET /identity/v3/role_assignments?group.id=b0fd84213ca0455aaf2983a2b994abb9&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:48.848695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ef33-5ac1-429f-87bc-92cbabdfedc1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:48.867413 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc6917-baf5-4e2a-9772-49f4019e45df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:48.869654 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc6917-baf5-4e2a-9772-49f4019e45df None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:48.930823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:48.983541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:48.987944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ef33-5ac1-429f-87bc-92cbabdfedc1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:49.004839 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 187/363] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:49.006279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deedb43-7a37-41d8-add9-6df033400eec None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.006566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deedb43-7a37-41d8-add9-6df033400eec None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.006844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deedb43-7a37-41d8-add9-6df033400eec None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.008459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 188/364] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:49.012817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deedb43-7a37-41d8-add9-6df033400eec None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:49.039822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7da2e-8a22-4338-9341-1158a7150a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:49.040889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7da2e-8a22-4338-9341-1158a7150a76 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.041173 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7da2e-8a22-4338-9341-1158a7150a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.041444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7da2e-8a22-4338-9341-1158a7150a76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.053580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7da2e-8a22-4338-9341-1158a7150a76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-86f7da2e-8a22-4338-9341-1158a7150a76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:49.062013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:49.062924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:49.063749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 177/365] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:49.064907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939c8f5-327b-4ba4-8e93-243c57b79da7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.068823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939c8f5-327b-4ba4-8e93-243c57b79da7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.068823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939c8f5-327b-4ba4-8e93-243c57b79da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.086104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1939c8f5-327b-4ba4-8e93-243c57b79da7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:49.087320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1939c8f5-327b-4ba4-8e93-243c57b79da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:49.147725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deedb43-7a37-41d8-add9-6df033400eec None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:49.166575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 178/366] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:49.167504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939c8f5-327b-4ba4-8e93-243c57b79da7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:49.192009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d6b2b-26f9-4453-8674-45f47489b452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:49.193034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d6b2b-26f9-4453-8674-45f47489b452 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.193319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d6b2b-26f9-4453-8674-45f47489b452 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.193903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d6b2b-26f9-4453-8674-45f47489b452 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.195390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d6b2b-26f9-4453-8674-45f47489b452 None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:49.197329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d6b2b-26f9-4453-8674-45f47489b452 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:49.199626 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 189/367] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:49 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:49.205240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d6b2b-26f9-4453-8674-45f47489b452 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:49.220825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:49.278368 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:49.291565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939c8f5-327b-4ba4-8e93-243c57b79da7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:49.329432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d6b2b-26f9-4453-8674-45f47489b452 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:49.332349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43234e0-02ec-40f6-b11e-a5993a8ac559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:49.333144 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43234e0-02ec-40f6-b11e-a5993a8ac559 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.333502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43234e0-02ec-40f6-b11e-a5993a8ac559 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.334085 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43234e0-02ec-40f6-b11e-a5993a8ac559 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.334852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43234e0-02ec-40f6-b11e-a5993a8ac559 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:49.335730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43234e0-02ec-40f6-b11e-a5993a8ac559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:49.337143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43234e0-02ec-40f6-b11e-a5993a8ac559 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:49.349627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 179/368] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:49.350623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43234e0-02ec-40f6-b11e-a5993a8ac559 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:49.380391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a41bf-497e-4c52-a881-63af3ec980e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:49.381200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a41bf-497e-4c52-a881-63af3ec980e1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.381484 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a41bf-497e-4c52-a881-63af3ec980e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.381759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a41bf-497e-4c52-a881-63af3ec980e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.393315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a41bf-497e-4c52-a881-63af3ec980e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-fe6a41bf-497e-4c52-a881-63af3ec980e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:49.401348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:49.402249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:49.403047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 190/369] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:49.403688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a41bf-497e-4c52-a881-63af3ec980e1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:49.463205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43234e0-02ec-40f6-b11e-a5993a8ac559 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:49.472530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:49.509279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0e4ec-785c-4793-9ed5-364b571f6571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:49.510323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0e4ec-785c-4793-9ed5-364b571f6571 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.510610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0e4ec-785c-4793-9ed5-364b571f6571 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.511186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0e4ec-785c-4793-9ed5-364b571f6571 None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.512374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0e4ec-785c-4793-9ed5-364b571f6571 None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:49.514265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0e4ec-785c-4793-9ed5-364b571f6571 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:49.518127 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 180/370] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:49 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:49.518990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0e4ec-785c-4793-9ed5-364b571f6571 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:49.536802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a41bf-497e-4c52-a881-63af3ec980e1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:49.558009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:49.582460 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92894fe-ac29-4ac0-bf46-98cddc5b985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:49.583289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92894fe-ac29-4ac0-bf46-98cddc5b985d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.583634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92894fe-ac29-4ac0-bf46-98cddc5b985d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.583964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92894fe-ac29-4ac0-bf46-98cddc5b985d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.635484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0e4ec-785c-4793-9ed5-364b571f6571 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:49.660888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92894fe-ac29-4ac0-bf46-98cddc5b985d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:49.662484 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92894fe-ac29-4ac0-bf46-98cddc5b985d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:49.673706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8d714-baee-41b6-9a13-3704b5e420a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:49.674412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8d714-baee-41b6-9a13-3704b5e420a7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.674776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8d714-baee-41b6-9a13-3704b5e420a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.675895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8d714-baee-41b6-9a13-3704b5e420a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.676651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8d714-baee-41b6-9a13-3704b5e420a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:49.676955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8d714-baee-41b6-9a13-3704b5e420a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:49.678468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8d714-baee-41b6-9a13-3704b5e420a7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:49.687624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 181/371] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:49.688577 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8d714-baee-41b6-9a13-3704b5e420a7 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:49.727335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:49.775291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 191/372] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:49.776113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92894fe-ac29-4ac0-bf46-98cddc5b985d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:49.792569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:49.799064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8d714-baee-41b6-9a13-3704b5e420a7 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:49.836755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf479f6b-7f71-480b-850a-786420dd1c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:49.837639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf479f6b-7f71-480b-850a-786420dd1c8c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.838077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf479f6b-7f71-480b-850a-786420dd1c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.838507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf479f6b-7f71-480b-850a-786420dd1c8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.839592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf479f6b-7f71-480b-850a-786420dd1c8c None admin] RBAC: Adding query filter params (group.id=b0fd84213ca0455aaf2983a2b994abb9, role.id=a70b63ede7014fa9bbcb4e03275a2664, scope.project.id=35d52ce6b9e5491abc31216be989797e) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:49.840132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf479f6b-7f71-480b-850a-786420dd1c8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:49.841684 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf479f6b-7f71-480b-850a-786420dd1c8c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:49.848603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 182/373] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 13 08:47:49 2022] GET /identity/v3/role_assignments?group.id=b0fd84213ca0455aaf2983a2b994abb9&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:49.849545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf479f6b-7f71-480b-850a-786420dd1c8c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:49.891681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92894fe-ac29-4ac0-bf46-98cddc5b985d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:49.932185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087235cc-5746-49f5-a13a-748051f8d789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:49.932822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087235cc-5746-49f5-a13a-748051f8d789 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:49.933108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087235cc-5746-49f5-a13a-748051f8d789 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:49.933384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087235cc-5746-49f5-a13a-748051f8d789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:49.964211 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:49.967323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf479f6b-7f71-480b-850a-786420dd1c8c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:50.005087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b365b9-42f6-426f-9629-a8a1b0c349ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:50.006249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b365b9-42f6-426f-9629-a8a1b0c349ec None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:50.006697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b365b9-42f6-426f-9629-a8a1b0c349ec None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:50.007260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b365b9-42f6-426f-9629-a8a1b0c349ec None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:50.008574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b365b9-42f6-426f-9629-a8a1b0c349ec None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:50.010181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b365b9-42f6-426f-9629-a8a1b0c349ec None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:50.012340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 183/374] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:49 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:50.017461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b365b9-42f6-426f-9629-a8a1b0c349ec None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:50.023904 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087235cc-5746-49f5-a13a-748051f8d789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:50.025402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087235cc-5746-49f5-a13a-748051f8d789 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:50.078465 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:50.190351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b365b9-42f6-426f-9629-a8a1b0c349ec None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:50.204092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 192/375] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:50.209917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087235cc-5746-49f5-a13a-748051f8d789 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:50.248625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c8028-d0ce-48a7-879b-bdea1e170ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:50.250542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c8028-d0ce-48a7-879b-bdea1e170ecd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:50.250830 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c8028-d0ce-48a7-879b-bdea1e170ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:50.251107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c8028-d0ce-48a7-879b-bdea1e170ecd None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:50.252412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c8028-d0ce-48a7-879b-bdea1e170ecd None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:50.254478 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c8028-d0ce-48a7-879b-bdea1e170ecd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:50.256711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 184/376] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:50 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:50.262582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c8028-d0ce-48a7-879b-bdea1e170ecd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:50.302575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:50.339855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087235cc-5746-49f5-a13a-748051f8d789 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:50.357063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:50.379140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c8028-d0ce-48a7-879b-bdea1e170ecd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:50.379844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32f8dd-9001-4e1e-a320-d2b275bfa0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:50.380844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32f8dd-9001-4e1e-a320-d2b275bfa0ed None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:50.381133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32f8dd-9001-4e1e-a320-d2b275bfa0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:50.381408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32f8dd-9001-4e1e-a320-d2b275bfa0ed None admin] PATH_INFO: `/v3/projects/c8052949c90742f6ada3f8a34d5636f8` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:50.382682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32f8dd-9001-4e1e-a320-d2b275bfa0ed None admin] RBAC: Authorizing `identity:get_project(project_id=c8052949c90742f6ada3f8a34d5636f8)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:50.384147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32f8dd-9001-4e1e-a320-d2b275bfa0ed None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:50.385858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 193/377] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:50 2022] GET /identity/v3/projects/c8052949c90742f6ada3f8a34d5636f8 => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:50.391422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32f8dd-9001-4e1e-a320-d2b275bfa0ed None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:50.419829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980dcb8-b25f-4fce-bd63-2a3c67e742a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:50.420490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980dcb8-b25f-4fce-bd63-2a3c67e742a9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:50.420827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980dcb8-b25f-4fce-bd63-2a3c67e742a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:50.421443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980dcb8-b25f-4fce-bd63-2a3c67e742a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:50.422418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980dcb8-b25f-4fce-bd63-2a3c67e742a9 None admin] RBAC: Adding query filter params (group.id=b0fd84213ca0455aaf2983a2b994abb9, role.id=120eeffad7094e10a69166d19aaa6439, scope.project.id=35d52ce6b9e5491abc31216be989797e) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:50.422729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980dcb8-b25f-4fce-bd63-2a3c67e742a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:50.424278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980dcb8-b25f-4fce-bd63-2a3c67e742a9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:50.430727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 185/378] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 13 08:47:50 2022] GET /identity/v3/role_assignments?group.id=b0fd84213ca0455aaf2983a2b994abb9&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=120eeffad7094e10a69166d19aaa6439 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:50.506026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32f8dd-9001-4e1e-a320-d2b275bfa0ed None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:50.545308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd9519-c63c-4178-b0eb-8408440ade63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:50.546467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd9519-c63c-4178-b0eb-8408440ade63 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:50.546748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd9519-c63c-4178-b0eb-8408440ade63 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:50.547023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd9519-c63c-4178-b0eb-8408440ade63 None admin] PATH_INFO: `/v3/groups/53768a0eacd3460c8f3df0771a018665` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:50.553166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd9519-c63c-4178-b0eb-8408440ade63 None admin] RBAC: Authorizing `identity:get_group(group_id=53768a0eacd3460c8f3df0771a018665)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:50.554710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd9519-c63c-4178-b0eb-8408440ade63 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:50.556859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 194/379] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:50 2022] GET /identity/v3/groups/53768a0eacd3460c8f3df0771a018665 => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:50.562396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980dcb8-b25f-4fce-bd63-2a3c67e742a9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:50.706648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:50.709687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980dcb8-b25f-4fce-bd63-2a3c67e742a9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:50.757011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd7671-d752-40f5-afe4-b1ba519f7c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:50.757766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd7671-d752-40f5-afe4-b1ba519f7c5f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:50.758081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd7671-d752-40f5-afe4-b1ba519f7c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:50.758454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd7671-d752-40f5-afe4-b1ba519f7c5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:50.759337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd7671-d752-40f5-afe4-b1ba519f7c5f None admin] RBAC: Adding query filter params (group.id=53768a0eacd3460c8f3df0771a018665, role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=c8052949c90742f6ada3f8a34d5636f8) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:50.759652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd7671-d752-40f5-afe4-b1ba519f7c5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:50.761202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd7671-d752-40f5-afe4-b1ba519f7c5f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:50.769780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 186/380] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 13 08:47:50 2022] GET /identity/v3/role_assignments?group.id=53768a0eacd3460c8f3df0771a018665&scope.project.id=c8052949c90742f6ada3f8a34d5636f8&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:51.326720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f51386-d2be-413b-b570-1e999f5fb9f9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:51.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f51386-d2be-413b-b570-1e999f5fb9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:51.327313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f51386-d2be-413b-b570-1e999f5fb9f9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:51.328757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 195/381] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:51.338333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b513d-5f64-4302-8004-7222dbdba9e9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:51.338333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b513d-5f64-4302-8004-7222dbdba9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:51.338333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b513d-5f64-4302-8004-7222dbdba9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:51.356774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d27b513d-5f64-4302-8004-7222dbdba9e9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:51.357975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d27b513d-5f64-4302-8004-7222dbdba9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:51.431342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6130041-0bf3-4495-82f7-3d0885813bfe None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:51.431685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6130041-0bf3-4495-82f7-3d0885813bfe None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:51.431890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6130041-0bf3-4495-82f7-3d0885813bfe None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:51.433259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 196/382] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:51.438555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81798f42-2fa0-4d4f-b830-be639557504a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:51.438837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81798f42-2fa0-4d4f-b830-be639557504a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:51.439123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81798f42-2fa0-4d4f-b830-be639557504a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:51.470784 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 187/383] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:51.471071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-81798f42-2fa0-4d4f-b830-be639557504a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:51.472281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-81798f42-2fa0-4d4f-b830-be639557504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:51.492576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b513d-5f64-4302-8004-7222dbdba9e9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:51.541420 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:51.585907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 197/384] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:51.625174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81798f42-2fa0-4d4f-b830-be639557504a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:51.653714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:51.665484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b513d-5f64-4302-8004-7222dbdba9e9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:51.735168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5941d-62fc-49b1-8b29-613adfec27eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:51.736217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5941d-62fc-49b1-8b29-613adfec27eb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:51.736500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5941d-62fc-49b1-8b29-613adfec27eb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:51.736778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5941d-62fc-49b1-8b29-613adfec27eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:51.763039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5941d-62fc-49b1-8b29-613adfec27eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-aac5941d-62fc-49b1-8b29-613adfec27eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:51.777717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:51.778733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:51.779562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:51.785365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 188/385] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:51.788532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5941d-62fc-49b1-8b29-613adfec27eb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:51.804700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81798f42-2fa0-4d4f-b830-be639557504a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:51.855422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b3c6c-a413-4020-b40c-5b6b77c3078b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:51.858830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b3c6c-a413-4020-b40c-5b6b77c3078b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:51.859280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b3c6c-a413-4020-b40c-5b6b77c3078b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:51.859737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b3c6c-a413-4020-b40c-5b6b77c3078b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:51.872297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b3c6c-a413-4020-b40c-5b6b77c3078b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-201b3c6c-a413-4020-b40c-5b6b77c3078b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:51.881124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:51.882230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:51.883217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:51.885734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 198/386] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:51.891489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b3c6c-a413-4020-b40c-5b6b77c3078b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:51.966457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5941d-62fc-49b1-8b29-613adfec27eb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:51.977889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:52.013061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2e738-3942-42ed-879a-14451a7f6528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:52.013738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e738-3942-42ed-879a-14451a7f6528 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:52.014070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e738-3942-42ed-879a-14451a7f6528 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:52.014799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e738-3942-42ed-879a-14451a7f6528 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:52.015562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2e738-3942-42ed-879a-14451a7f6528 None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:52.015873 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2e738-3942-42ed-879a-14451a7f6528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:52.017571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2e738-3942-42ed-879a-14451a7f6528 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:52.038331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 189/387] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:52.045597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2e738-3942-42ed-879a-14451a7f6528 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:52.087724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:52.109412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b3c6c-a413-4020-b40c-5b6b77c3078b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:52.156291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97be07-fa98-419d-8e17-a900d4c40655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:52.156966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97be07-fa98-419d-8e17-a900d4c40655 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:52.157247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97be07-fa98-419d-8e17-a900d4c40655 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:52.157524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97be07-fa98-419d-8e17-a900d4c40655 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:52.158296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97be07-fa98-419d-8e17-a900d4c40655 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:52.158599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97be07-fa98-419d-8e17-a900d4c40655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:52.160991 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97be07-fa98-419d-8e17-a900d4c40655 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:52.170087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 199/388] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:52.177659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97be07-fa98-419d-8e17-a900d4c40655 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:52.289563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:52.294399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97be07-fa98-419d-8e17-a900d4c40655 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:52.308271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2e738-3942-42ed-879a-14451a7f6528 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:52.340519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c182fba-6727-4549-b55d-48270ffa1e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:52.342080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c182fba-6727-4549-b55d-48270ffa1e0e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:52.342689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c182fba-6727-4549-b55d-48270ffa1e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:52.343111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c182fba-6727-4549-b55d-48270ffa1e0e None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:52.344486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c182fba-6727-4549-b55d-48270ffa1e0e None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:52.346886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c182fba-6727-4549-b55d-48270ffa1e0e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:52.349719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 200/389] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:52 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:52.360853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c182fba-6727-4549-b55d-48270ffa1e0e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:52.371566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528ff14-c6bc-42ad-9a3b-b0f1bc81df73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:52.372543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528ff14-c6bc-42ad-9a3b-b0f1bc81df73 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:52.372895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528ff14-c6bc-42ad-9a3b-b0f1bc81df73 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:52.373237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528ff14-c6bc-42ad-9a3b-b0f1bc81df73 None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:52.378597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528ff14-c6bc-42ad-9a3b-b0f1bc81df73 None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:52.378597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528ff14-c6bc-42ad-9a3b-b0f1bc81df73 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:52.378597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 190/390] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:52 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:52.385658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528ff14-c6bc-42ad-9a3b-b0f1bc81df73 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:52.528777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c182fba-6727-4549-b55d-48270ffa1e0e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:52.548506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:52.567323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528ff14-c6bc-42ad-9a3b-b0f1bc81df73 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:52.585488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f98abf-2e52-487a-a522-59b06554a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:52.586380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f98abf-2e52-487a-a522-59b06554a8cc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:52.586929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f98abf-2e52-487a-a522-59b06554a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:52.586929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f98abf-2e52-487a-a522-59b06554a8cc None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:52.588826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f98abf-2e52-487a-a522-59b06554a8cc None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:52.590314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f98abf-2e52-487a-a522-59b06554a8cc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:52.592292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 201/391] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:52 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:52.593563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eba695-045f-4881-834a-9776c925e045 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:52.593846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eba695-045f-4881-834a-9776c925e045 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:52.598312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eba695-045f-4881-834a-9776c925e045 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:52.599836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 202/392] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:52.605270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a20daa-5888-4c4e-a97c-6504d03c413b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:52.605506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a20daa-5888-4c4e-a97c-6504d03c413b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:52.605744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a20daa-5888-4c4e-a97c-6504d03c413b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:52.618907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77246490-5deb-4b96-8ae1-08d41329caf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:52.619700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77246490-5deb-4b96-8ae1-08d41329caf8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:52.619979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77246490-5deb-4b96-8ae1-08d41329caf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:52.620250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77246490-5deb-4b96-8ae1-08d41329caf8 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:52.624292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77246490-5deb-4b96-8ae1-08d41329caf8 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:52.624292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77246490-5deb-4b96-8ae1-08d41329caf8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:52.630126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 191/393] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:52 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:52.631254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77246490-5deb-4b96-8ae1-08d41329caf8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:52.632510 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-18a20daa-5888-4c4e-a97c-6504d03c413b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:52.633919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-18a20daa-5888-4c4e-a97c-6504d03c413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:52.673625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:52.726218 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 203/394] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:52.727459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a20daa-5888-4c4e-a97c-6504d03c413b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:52.772404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77246490-5deb-4b96-8ae1-08d41329caf8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:52.813945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09131d-340f-4e4c-a59b-d9c3bf7c53e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:52.814870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09131d-340f-4e4c-a59b-d9c3bf7c53e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:52.815148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09131d-340f-4e4c-a59b-d9c3bf7c53e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:52.816006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09131d-340f-4e4c-a59b-d9c3bf7c53e4 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418/groups/b0fd84213ca0455aaf2983a2b994abb9/roles/120eeffad7094e10a69166d19aaa6439` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:52.818374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09131d-340f-4e4c-a59b-d9c3bf7c53e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=245d48325d884a83b0a57436df3e1418, group_id=b0fd84213ca0455aaf2983a2b994abb9, role_id=120eeffad7094e10a69166d19aaa6439)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:52.819803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09131d-340f-4e4c-a59b-d9c3bf7c53e4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:52.861410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a20daa-5888-4c4e-a97c-6504d03c413b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:52.901848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 192/395] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 13 08:47:52 2022] PUT /identity/v3/projects/245d48325d884a83b0a57436df3e1418/groups/b0fd84213ca0455aaf2983a2b994abb9/roles/120eeffad7094e10a69166d19aaa6439 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:52.903058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09131d-340f-4e4c-a59b-d9c3bf7c53e4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:52.911745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ca8c5-15a6-46f9-bd41-9065094c1b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:52.915751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ca8c5-15a6-46f9-bd41-9065094c1b40 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:52.916738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ca8c5-15a6-46f9-bd41-9065094c1b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:52.917658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ca8c5-15a6-46f9-bd41-9065094c1b40 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418/groups/b0fd84213ca0455aaf2983a2b994abb9/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:52.923669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ca8c5-15a6-46f9-bd41-9065094c1b40 None admin] RBAC: Authorizing `identity:create_grant(project_id=245d48325d884a83b0a57436df3e1418, group_id=b0fd84213ca0455aaf2983a2b994abb9, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:52.927831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ca8c5-15a6-46f9-bd41-9065094c1b40 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:52.941199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:53.025070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 204/396] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 13 08:47:52 2022] PUT /identity/v3/projects/245d48325d884a83b0a57436df3e1418/groups/b0fd84213ca0455aaf2983a2b994abb9/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:53.056804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09131d-340f-4e4c-a59b-d9c3bf7c53e4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:53.099092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18161f-6141-40c0-b610-3e2580b50505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:53.100008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18161f-6141-40c0-b610-3e2580b50505 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.100344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18161f-6141-40c0-b610-3e2580b50505 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.100632 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18161f-6141-40c0-b610-3e2580b50505 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.103917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef625-191d-44b8-8bbb-0285889ad6b6 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.104218 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef625-191d-44b8-8bbb-0285889ad6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.104502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef625-191d-44b8-8bbb-0285889ad6b6 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.106203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 205/397] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:53.112768 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea379a8-aef8-47ab-8c4b-8dae6bdcee17 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.113359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea379a8-aef8-47ab-8c4b-8dae6bdcee17 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.113856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea379a8-aef8-47ab-8c4b-8dae6bdcee17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.124003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18161f-6141-40c0-b610-3e2580b50505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-1e18161f-6141-40c0-b610-3e2580b50505 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:53.133333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:53.134434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:53.135265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:53.137391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 193/398] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:53.144735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18161f-6141-40c0-b610-3e2580b50505 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:53.145303 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6ea379a8-aef8-47ab-8c4b-8dae6bdcee17 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:53.147161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6ea379a8-aef8-47ab-8c4b-8dae6bdcee17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:53.169877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:53.257905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:53.282499 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 206/399] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:47:53.301893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18161f-6141-40c0-b610-3e2580b50505 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:53.308610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea379a8-aef8-47ab-8c4b-8dae6bdcee17 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:53.354376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b59e71-5d56-449a-89e5-fb721026176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:53.355049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b59e71-5d56-449a-89e5-fb721026176b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.355324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b59e71-5d56-449a-89e5-fb721026176b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.355589 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b59e71-5d56-449a-89e5-fb721026176b None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.356327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b59e71-5d56-449a-89e5-fb721026176b None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:53.356628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b59e71-5d56-449a-89e5-fb721026176b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:53.357990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b59e71-5d56-449a-89e5-fb721026176b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:53.368262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 194/400] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:53.373597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b59e71-5d56-449a-89e5-fb721026176b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:53.442062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea379a8-aef8-47ab-8c4b-8dae6bdcee17 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:53.479638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297f888-5828-465b-800b-f8ed79a2a523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:53.480821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297f888-5828-465b-800b-f8ed79a2a523 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.481302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297f888-5828-465b-800b-f8ed79a2a523 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.481786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297f888-5828-465b-800b-f8ed79a2a523 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.493853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297f888-5828-465b-800b-f8ed79a2a523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5297f888-5828-465b-800b-f8ed79a2a523 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:53.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:53.504405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:53.505244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:53.512530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 207/401] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:53.513954 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297f888-5828-465b-800b-f8ed79a2a523 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:53.531980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:53.629149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297f888-5828-465b-800b-f8ed79a2a523 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:53.633050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b59e71-5d56-449a-89e5-fb721026176b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:53.668081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed455310-952f-491e-a4e2-4bd9ff02e17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:53.669226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed455310-952f-491e-a4e2-4bd9ff02e17d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.669671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed455310-952f-491e-a4e2-4bd9ff02e17d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.670113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed455310-952f-491e-a4e2-4bd9ff02e17d None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.670222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c6eea-8370-41ca-bbee-2dcdd47e67b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:53.671289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c6eea-8370-41ca-bbee-2dcdd47e67b3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.671515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed455310-952f-491e-a4e2-4bd9ff02e17d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:53.671638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c6eea-8370-41ca-bbee-2dcdd47e67b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.671894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c6eea-8370-41ca-bbee-2dcdd47e67b3 None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.672221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed455310-952f-491e-a4e2-4bd9ff02e17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:53.673058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c6eea-8370-41ca-bbee-2dcdd47e67b3 None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:53.673904 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed455310-952f-491e-a4e2-4bd9ff02e17d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:53.674714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c6eea-8370-41ca-bbee-2dcdd47e67b3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:53.677934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 195/402] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:53 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:53.684232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b89b25-3afc-48fa-b9e4-792a944e90b9 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.684517 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b89b25-3afc-48fa-b9e4-792a944e90b9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.684790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b89b25-3afc-48fa-b9e4-792a944e90b9 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.686224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 196/403] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:53.687014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b89b25-3afc-48fa-b9e4-792a944e90b9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:53.689132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 208/404] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:53.697588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:53.697726 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfde835-24c3-4b86-b559-f175ba202db3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.698077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfde835-24c3-4b86-b559-f175ba202db3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.698980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfde835-24c3-4b86-b559-f175ba202db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.716514 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-dcfde835-24c3-4b86-b559-f175ba202db3 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:53.717809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dcfde835-24c3-4b86-b559-f175ba202db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:53.795782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 209/405] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:53.797258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfde835-24c3-4b86-b559-f175ba202db3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:53.806372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:53.849525 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b89b25-3afc-48fa-b9e4-792a944e90b9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:53.883809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b259b34-71da-49a0-b544-868cd4ec8d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:53.884680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259b34-71da-49a0-b544-868cd4ec8d3a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.884961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259b34-71da-49a0-b544-868cd4ec8d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.885253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259b34-71da-49a0-b544-868cd4ec8d3a None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.886477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b259b34-71da-49a0-b544-868cd4ec8d3a None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:53.887884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b259b34-71da-49a0-b544-868cd4ec8d3a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:53.889728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 197/406] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:53 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:53.894769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b259b34-71da-49a0-b544-868cd4ec8d3a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:53.913798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfde835-24c3-4b86-b559-f175ba202db3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:53.949364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbc145-713a-4167-8592-29180c2e302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:53.950615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbc145-713a-4167-8592-29180c2e302b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:53.951107 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbc145-713a-4167-8592-29180c2e302b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:53.951568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbc145-713a-4167-8592-29180c2e302b None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:53.952955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbc145-713a-4167-8592-29180c2e302b None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:53.954645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbc145-713a-4167-8592-29180c2e302b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:53.961680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 210/407] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:53 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:53.962623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:53.963050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbc145-713a-4167-8592-29180c2e302b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:53.998205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b259b34-71da-49a0-b544-868cd4ec8d3a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:54.032919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd21aa-4306-40a9-a3dc-6339b15cdfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:54.033715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd21aa-4306-40a9-a3dc-6339b15cdfee None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:54.034593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd21aa-4306-40a9-a3dc-6339b15cdfee None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:54.035376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd21aa-4306-40a9-a3dc-6339b15cdfee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:54.044166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:54.045826 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd21aa-4306-40a9-a3dc-6339b15cdfee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d4bd21aa-4306-40a9-a3dc-6339b15cdfee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:54.053862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:54.054811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:54.055594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 198/408] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:54.056270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd21aa-4306-40a9-a3dc-6339b15cdfee None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:54.074122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbc145-713a-4167-8592-29180c2e302b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:54.111673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c71360-2cbf-40ab-8293-96ee2888f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:54.112820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c71360-2cbf-40ab-8293-96ee2888f628 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:54.113238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c71360-2cbf-40ab-8293-96ee2888f628 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:54.113667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c71360-2cbf-40ab-8293-96ee2888f628 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e/groups/b0fd84213ca0455aaf2983a2b994abb9/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:54.118098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c71360-2cbf-40ab-8293-96ee2888f628 None admin] RBAC: Authorizing `identity:create_grant(project_id=35d52ce6b9e5491abc31216be989797e, group_id=b0fd84213ca0455aaf2983a2b994abb9, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:54.119721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c71360-2cbf-40ab-8293-96ee2888f628 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:54.168285 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd21aa-4306-40a9-a3dc-6339b15cdfee None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:54.197652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:54.203255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d27f57-0248-4f4e-84fd-e24c11f8b5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:54.204063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d27f57-0248-4f4e-84fd-e24c11f8b5cf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:54.204338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d27f57-0248-4f4e-84fd-e24c11f8b5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:54.204607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d27f57-0248-4f4e-84fd-e24c11f8b5cf None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:54.205825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d27f57-0248-4f4e-84fd-e24c11f8b5cf None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:54.210403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d27f57-0248-4f4e-84fd-e24c11f8b5cf None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:54.214917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 211/409] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 13 08:47:53 2022] PUT /identity/v3/projects/35d52ce6b9e5491abc31216be989797e/groups/b0fd84213ca0455aaf2983a2b994abb9/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:54.215524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c71360-2cbf-40ab-8293-96ee2888f628 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:54.216176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 199/410] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:54 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:54.230155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d27f57-0248-4f4e-84fd-e24c11f8b5cf None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:54.361448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d27f57-0248-4f4e-84fd-e24c11f8b5cf None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:54.407017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e1bb5-2ba0-4bf4-baa9-65e3e134cde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:54.408352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e1bb5-2ba0-4bf4-baa9-65e3e134cde6 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:54.409536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e1bb5-2ba0-4bf4-baa9-65e3e134cde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:54.410347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e1bb5-2ba0-4bf4-baa9-65e3e134cde6 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e/groups/b0fd84213ca0455aaf2983a2b994abb9/roles/120eeffad7094e10a69166d19aaa6439` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:54.413811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e1bb5-2ba0-4bf4-baa9-65e3e134cde6 None admin] RBAC: Authorizing `identity:create_grant(project_id=35d52ce6b9e5491abc31216be989797e, group_id=b0fd84213ca0455aaf2983a2b994abb9, role_id=120eeffad7094e10a69166d19aaa6439)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:54.415561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e1bb5-2ba0-4bf4-baa9-65e3e134cde6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:54.453093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c71360-2cbf-40ab-8293-96ee2888f628 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:54.492028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24eaca7-56b3-4f7c-b80e-88fd437e42db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:54.492691 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24eaca7-56b3-4f7c-b80e-88fd437e42db None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:54.492973 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24eaca7-56b3-4f7c-b80e-88fd437e42db None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:54.493244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24eaca7-56b3-4f7c-b80e-88fd437e42db None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:54.494006 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24eaca7-56b3-4f7c-b80e-88fd437e42db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:54.498094 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24eaca7-56b3-4f7c-b80e-88fd437e42db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:54.498094 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24eaca7-56b3-4f7c-b80e-88fd437e42db None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:54.504461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 200/411] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 13 08:47:54 2022] PUT /identity/v3/projects/35d52ce6b9e5491abc31216be989797e/groups/b0fd84213ca0455aaf2983a2b994abb9/roles/120eeffad7094e10a69166d19aaa6439 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:54.510071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:54.510071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 212/412] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:54.514227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e1bb5-2ba0-4bf4-baa9-65e3e134cde6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:54.588745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:54.637874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e1bb5-2ba0-4bf4-baa9-65e3e134cde6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:54.679901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae72bc-5742-48ae-914c-fdaca6c9e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:54.680765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae72bc-5742-48ae-914c-fdaca6c9e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:54.681107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae72bc-5742-48ae-914c-fdaca6c9e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:54.681385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae72bc-5742-48ae-914c-fdaca6c9e1b0 None admin] PATH_INFO: `/v3/groups/53768a0eacd3460c8f3df0771a018665` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:54.682631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae72bc-5742-48ae-914c-fdaca6c9e1b0 None admin] RBAC: Authorizing `identity:get_group(group_id=53768a0eacd3460c8f3df0771a018665)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:54.686456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae72bc-5742-48ae-914c-fdaca6c9e1b0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:54.691887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 201/413] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:54 2022] GET /identity/v3/groups/53768a0eacd3460c8f3df0771a018665 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:54.695628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24eaca7-56b3-4f7c-b80e-88fd437e42db None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:54.808746 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24eaca7-56b3-4f7c-b80e-88fd437e42db None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:54.845206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bbcdb-610f-4f7a-81f6-bf6fb3e47cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:54.846094 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bbcdb-610f-4f7a-81f6-bf6fb3e47cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:54.847135 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bbcdb-610f-4f7a-81f6-bf6fb3e47cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:54.847887 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bbcdb-610f-4f7a-81f6-bf6fb3e47cc7 None admin] PATH_INFO: `/v3/projects/c8052949c90742f6ada3f8a34d5636f8` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:54.849098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bbcdb-610f-4f7a-81f6-bf6fb3e47cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=c8052949c90742f6ada3f8a34d5636f8)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:54.850638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bbcdb-610f-4f7a-81f6-bf6fb3e47cc7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:54.852373 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 213/414] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:54 2022] GET /identity/v3/projects/c8052949c90742f6ada3f8a34d5636f8 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:54.858209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae72bc-5742-48ae-914c-fdaca6c9e1b0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:54.969995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae72bc-5742-48ae-914c-fdaca6c9e1b0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:55.007010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:55.016224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cdf56-ea51-4645-acd3-259d6a283af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:55.017111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cdf56-ea51-4645-acd3-259d6a283af8 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:55.017391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cdf56-ea51-4645-acd3-259d6a283af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:55.017664 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cdf56-ea51-4645-acd3-259d6a283af8 None admin] PATH_INFO: `/v3/projects/c8052949c90742f6ada3f8a34d5636f8/groups/53768a0eacd3460c8f3df0771a018665/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:55.020836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cdf56-ea51-4645-acd3-259d6a283af8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8052949c90742f6ada3f8a34d5636f8, group_id=53768a0eacd3460c8f3df0771a018665, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:55.022296 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cdf56-ea51-4645-acd3-259d6a283af8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:55.111653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 202/415] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 13 08:47:54 2022] PUT /identity/v3/projects/c8052949c90742f6ada3f8a34d5636f8/groups/53768a0eacd3460c8f3df0771a018665/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:47:55.571154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dda4fc-f347-45d5-9f5c-25b8a9b2882f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:55.571493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dda4fc-f347-45d5-9f5c-25b8a9b2882f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:55.571677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dda4fc-f347-45d5-9f5c-25b8a9b2882f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:55.573257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 214/416] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:55.587572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd26380-ba5f-43f7-9c22-0791b75371b0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:55.587865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd26380-ba5f-43f7-9c22-0791b75371b0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:55.588148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd26380-ba5f-43f7-9c22-0791b75371b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:55.611056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5bd26380-ba5f-43f7-9c22-0791b75371b0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:55.613550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5bd26380-ba5f-43f7-9c22-0791b75371b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:55.619060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88966417-2ab3-4b57-b78b-1344dcd4d062 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:55.619345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88966417-2ab3-4b57-b78b-1344dcd4d062 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:55.619622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88966417-2ab3-4b57-b78b-1344dcd4d062 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:55.621019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 215/417] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:55.625788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a80db-86c0-414a-b37e-0516bea8f2b9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:55.626084 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a80db-86c0-414a-b37e-0516bea8f2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:55.626363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a80db-86c0-414a-b37e-0516bea8f2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:55.661557 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-190a80db-86c0-414a-b37e-0516bea8f2b9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:55.662771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-190a80db-86c0-414a-b37e-0516bea8f2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:55.744903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 203/418] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:55.787596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd26380-ba5f-43f7-9c22-0791b75371b0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:55.791083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:55.808931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 216/419] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 13 08:47:55.859574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a80db-86c0-414a-b37e-0516bea8f2b9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:55.959652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd26380-ba5f-43f7-9c22-0791b75371b0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:55.999262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:56.022294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d48f8-9713-4874-a45e-0e9ea9a086fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:56.023108 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d48f8-9713-4874-a45e-0e9ea9a086fb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:56.023384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d48f8-9713-4874-a45e-0e9ea9a086fb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:56.023655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d48f8-9713-4874-a45e-0e9ea9a086fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:56.035071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a80db-86c0-414a-b37e-0516bea8f2b9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:56.048023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d48f8-9713-4874-a45e-0e9ea9a086fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ab9d48f8-9713-4874-a45e-0e9ea9a086fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:56.059849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:56.060731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:56.061546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:47:56.062348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 204/420] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:47:56.071770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d48f8-9713-4874-a45e-0e9ea9a086fb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:56.092535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30968d-67ea-473e-93f9-b2d4950e90b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:56.093583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30968d-67ea-473e-93f9-b2d4950e90b7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:56.094062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30968d-67ea-473e-93f9-b2d4950e90b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:56.094538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30968d-67ea-473e-93f9-b2d4950e90b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:56.109468 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e30968d-67ea-473e-93f9-b2d4950e90b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3e30968d-67ea-473e-93f9-b2d4950e90b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:56.118590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:56.119516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:56.120323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:56.123309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 217/421] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:56.133613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30968d-67ea-473e-93f9-b2d4950e90b7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:56.229187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:56.264811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d48f8-9713-4874-a45e-0e9ea9a086fb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:56.308775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30968d-67ea-473e-93f9-b2d4950e90b7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:56.341886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec141c-0e4b-4f67-9a57-6fcd951a0403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:56.342589 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec141c-0e4b-4f67-9a57-6fcd951a0403 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:56.342867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec141c-0e4b-4f67-9a57-6fcd951a0403 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:56.343140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec141c-0e4b-4f67-9a57-6fcd951a0403 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:56.343883 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec141c-0e4b-4f67-9a57-6fcd951a0403 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:56.344187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec141c-0e4b-4f67-9a57-6fcd951a0403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:56.345547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec141c-0e4b-4f67-9a57-6fcd951a0403 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:56.350288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60f46a-7dcc-457a-9dc3-0b4d134b7b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:56.350949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f46a-7dcc-457a-9dc3-0b4d134b7b85 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:56.351226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f46a-7dcc-457a-9dc3-0b4d134b7b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:56.351498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f46a-7dcc-457a-9dc3-0b4d134b7b85 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:56.352240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60f46a-7dcc-457a-9dc3-0b4d134b7b85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:56.352543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60f46a-7dcc-457a-9dc3-0b4d134b7b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:56.353898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60f46a-7dcc-457a-9dc3-0b4d134b7b85 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:56.363238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 218/422] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:56.369434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60f46a-7dcc-457a-9dc3-0b4d134b7b85 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:56.373954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 205/423] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:56.381619 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec141c-0e4b-4f67-9a57-6fcd951a0403 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:56.445136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:56.513875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60f46a-7dcc-457a-9dc3-0b4d134b7b85 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:56.516269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec141c-0e4b-4f67-9a57-6fcd951a0403 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:56.535237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:56.552308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6679383-52cd-4caa-9d95-523cd832cd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:56.553094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6679383-52cd-4caa-9d95-523cd832cd19 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:56.553445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6679383-52cd-4caa-9d95-523cd832cd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:56.553792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6679383-52cd-4caa-9d95-523cd832cd19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:56.559130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2af955-382c-41b1-9550-505a67d1b115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:56.560007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2af955-382c-41b1-9550-505a67d1b115 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:56.560473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2af955-382c-41b1-9550-505a67d1b115 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:56.560923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2af955-382c-41b1-9550-505a67d1b115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:56.642122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2af955-382c-41b1-9550-505a67d1b115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:56.643615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2af955-382c-41b1-9550-505a67d1b115 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:56.644519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6679383-52cd-4caa-9d95-523cd832cd19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:56.647313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6679383-52cd-4caa-9d95-523cd832cd19 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:56.703137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:56.756619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 219/424] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:56.758346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a7144-2c87-454b-bfe5-3ac9567a3563 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:56.758828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a7144-2c87-454b-bfe5-3ac9567a3563 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:56.759335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a7144-2c87-454b-bfe5-3ac9567a3563 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:56.764792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 220/425] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:56.765982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a7144-2c87-454b-bfe5-3ac9567a3563 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:56.786387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 206/426] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:56.787215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:56.789224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c970c5-e254-4598-8525-eabf8fdd6dcf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:56.789507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c970c5-e254-4598-8525-eabf8fdd6dcf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:56.789780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c970c5-e254-4598-8525-eabf8fdd6dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:56.810924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f8c970c5-e254-4598-8525-eabf8fdd6dcf None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:56.812101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f8c970c5-e254-4598-8525-eabf8fdd6dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:56.889734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a7144-2c87-454b-bfe5-3ac9567a3563 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:56.898025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 207/427] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:56.899610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c970c5-e254-4598-8525-eabf8fdd6dcf None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:56.936120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08c58b-5855-4378-8de4-57f1a4672504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:56.936989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08c58b-5855-4378-8de4-57f1a4672504 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:56.937268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08c58b-5855-4378-8de4-57f1a4672504 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:56.937541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08c58b-5855-4378-8de4-57f1a4672504 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:56.938888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08c58b-5855-4378-8de4-57f1a4672504 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:56.940403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08c58b-5855-4378-8de4-57f1a4672504 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:56.942246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 221/428] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:56 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:56.942849 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08c58b-5855-4378-8de4-57f1a4672504 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:57.012789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:57.035008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:57.053684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08c58b-5855-4378-8de4-57f1a4672504 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:57.057443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c970c5-e254-4598-8525-eabf8fdd6dcf None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:57.094447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948f759-0e6a-4e49-96a0-cd684f07e184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:57.095509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948f759-0e6a-4e49-96a0-cd684f07e184 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.095883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948f759-0e6a-4e49-96a0-cd684f07e184 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.097075 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948f759-0e6a-4e49-96a0-cd684f07e184 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.104243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77074df9-ca30-4cfe-8271-e5988d716381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:57.105453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77074df9-ca30-4cfe-8271-e5988d716381 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.106174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77074df9-ca30-4cfe-8271-e5988d716381 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.106607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77074df9-ca30-4cfe-8271-e5988d716381 None admin] PATH_INFO: `/v3/projects/245d48325d884a83b0a57436df3e1418` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.108062 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77074df9-ca30-4cfe-8271-e5988d716381 None admin] RBAC: Authorizing `identity:get_project(project_id=245d48325d884a83b0a57436df3e1418)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:57.108841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948f759-0e6a-4e49-96a0-cd684f07e184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:57.109908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77074df9-ca30-4cfe-8271-e5988d716381 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:57.111751 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 208/429] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:56 2022] GET /identity/v3/projects/245d48325d884a83b0a57436df3e1418 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:57.112678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77074df9-ca30-4cfe-8271-e5988d716381 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e948f759-0e6a-4e49-96a0-cd684f07e184 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:57.118432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:57.119494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 08:47:57.120473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:57.127096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 222/430] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:57.127096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948f759-0e6a-4e49-96a0-cd684f07e184 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:57.236767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77074df9-ca30-4cfe-8271-e5988d716381 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:57.241644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948f759-0e6a-4e49-96a0-cd684f07e184 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:57.277058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d01524-85fb-4ace-befa-bdbd302b1d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:57.277902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d01524-85fb-4ace-befa-bdbd302b1d41 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.278304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d01524-85fb-4ace-befa-bdbd302b1d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.278537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94caa05-705e-42fb-b45e-dd305e84d2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:57.278649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d01524-85fb-4ace-befa-bdbd302b1d41 None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.279380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94caa05-705e-42fb-b45e-dd305e84d2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.279664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94caa05-705e-42fb-b45e-dd305e84d2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.279764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d01524-85fb-4ace-befa-bdbd302b1d41 None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:57.279932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94caa05-705e-42fb-b45e-dd305e84d2e5 None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.281103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94caa05-705e-42fb-b45e-dd305e84d2e5 None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:57.281197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d01524-85fb-4ace-befa-bdbd302b1d41 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:57.282631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94caa05-705e-42fb-b45e-dd305e84d2e5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:57.283089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 209/431] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:57 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:57.283965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d01524-85fb-4ace-befa-bdbd302b1d41 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:57.288876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:57.289339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 223/432] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:57 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:57.290611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40525f2c-8b38-4fbd-910d-a91c138fc35f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.290901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40525f2c-8b38-4fbd-910d-a91c138fc35f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.291182 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40525f2c-8b38-4fbd-910d-a91c138fc35f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.292393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 224/433] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:57.294183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40525f2c-8b38-4fbd-910d-a91c138fc35f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:57.313170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:57.398497 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d01524-85fb-4ace-befa-bdbd302b1d41 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:57.414165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40525f2c-8b38-4fbd-910d-a91c138fc35f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:57.441210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f805985-9dca-4009-8af0-0718b4ea8387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:57.442229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f805985-9dca-4009-8af0-0718b4ea8387 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.442653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f805985-9dca-4009-8af0-0718b4ea8387 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.442929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f805985-9dca-4009-8af0-0718b4ea8387 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.443667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f805985-9dca-4009-8af0-0718b4ea8387 None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:57.443967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f805985-9dca-4009-8af0-0718b4ea8387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:57.445327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f805985-9dca-4009-8af0-0718b4ea8387 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:57.451160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40458d-aeaa-43d8-8ee8-76b7e1164db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:57.452280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40458d-aeaa-43d8-8ee8-76b7e1164db3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.452758 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40458d-aeaa-43d8-8ee8-76b7e1164db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.453206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40458d-aeaa-43d8-8ee8-76b7e1164db3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.453682 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 210/434] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:47:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:57.454451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40458d-aeaa-43d8-8ee8-76b7e1164db3 None admin] RBAC: Adding query filter params (group.id=b0fd84213ca0455aaf2983a2b994abb9, role.id=a70b63ede7014fa9bbcb4e03275a2664, scope.project.id=245d48325d884a83b0a57436df3e1418) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:57.454940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40458d-aeaa-43d8-8ee8-76b7e1164db3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:57.455057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9d40a-d077-4d0a-af3b-122cddd60780 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.455225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9d40a-d077-4d0a-af3b-122cddd60780 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.455505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9d40a-d077-4d0a-af3b-122cddd60780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.457039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40458d-aeaa-43d8-8ee8-76b7e1164db3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:57.465110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 225/435] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 13 08:47:57 2022] GET /identity/v3/role_assignments?group.id=b0fd84213ca0455aaf2983a2b994abb9&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:57.466177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40458d-aeaa-43d8-8ee8-76b7e1164db3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:57.478313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a9a9d40a-d077-4d0a-af3b-122cddd60780 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:57.479516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a9a9d40a-d077-4d0a-af3b-122cddd60780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:57.556137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:57.564470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 211/436] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:57.566267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9d40a-d077-4d0a-af3b-122cddd60780 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:57.584883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:57.609763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40458d-aeaa-43d8-8ee8-76b7e1164db3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:57.646794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fda07-6e31-4362-9b78-4ac0d323c3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:57.647454 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fda07-6e31-4362-9b78-4ac0d323c3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.647738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fda07-6e31-4362-9b78-4ac0d323c3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.648021 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fda07-6e31-4362-9b78-4ac0d323c3c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.648855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fda07-6e31-4362-9b78-4ac0d323c3c9 None admin] RBAC: Adding query filter params (group.id=b0fd84213ca0455aaf2983a2b994abb9, role.id=120eeffad7094e10a69166d19aaa6439, scope.project.id=245d48325d884a83b0a57436df3e1418) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:57.649168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fda07-6e31-4362-9b78-4ac0d323c3c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:57.650649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fda07-6e31-4362-9b78-4ac0d323c3c9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:57.657811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 226/437] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 13 08:47:57 2022] GET /identity/v3/role_assignments?group.id=b0fd84213ca0455aaf2983a2b994abb9&scope.project.id=245d48325d884a83b0a57436df3e1418&role.id=120eeffad7094e10a69166d19aaa6439 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:57.659773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf537415-6825-4d30-839d-d60df59c80c1 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.660062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf537415-6825-4d30-839d-d60df59c80c1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.660343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf537415-6825-4d30-839d-d60df59c80c1 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.661600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 227/438] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:47:57.662331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf537415-6825-4d30-839d-d60df59c80c1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:57.697980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9d40a-d077-4d0a-af3b-122cddd60780 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:57.757523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc22657-dfef-4ef0-91a6-ed62210995fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:57.758225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc22657-dfef-4ef0-91a6-ed62210995fb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.758505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc22657-dfef-4ef0-91a6-ed62210995fb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.758778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc22657-dfef-4ef0-91a6-ed62210995fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.814881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf537415-6825-4d30-839d-d60df59c80c1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:57.844593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:57.853420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc22657-dfef-4ef0-91a6-ed62210995fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:57.855010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc22657-dfef-4ef0-91a6-ed62210995fb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:57.861188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b4aed-5897-4749-ac81-317d4915aaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:57.866703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b4aed-5897-4749-ac81-317d4915aaec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.866703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b4aed-5897-4749-ac81-317d4915aaec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.866703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b4aed-5897-4749-ac81-317d4915aaec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.874542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:57.880017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b4aed-5897-4749-ac81-317d4915aaec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-774b4aed-5897-4749-ac81-317d4915aaec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:57.890278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:57.891605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:47:57.892403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:57.895664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 228/439] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:47:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:57.897128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6548d-c014-487e-a185-b703b052a291 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:57.897629 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6548d-c014-487e-a185-b703b052a291 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:57.897912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6548d-c014-487e-a185-b703b052a291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:57.916138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d1c6548d-c014-487e-a185-b703b052a291 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:47:57.917367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d1c6548d-c014-487e-a185-b703b052a291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:47:57.986326 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 212/440] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:57.987796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc22657-dfef-4ef0-91a6-ed62210995fb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:57.993018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 229/441] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:47:57.993863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6548d-c014-487e-a185-b703b052a291 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:58.110687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:58.113011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6548d-c014-487e-a185-b703b052a291 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:58.117405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc22657-dfef-4ef0-91a6-ed62210995fb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:58.142612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:58.148327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730a56c-bb85-4951-bd0d-fe716bb6aba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:58.149192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730a56c-bb85-4951-bd0d-fe716bb6aba8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:58.149469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730a56c-bb85-4951-bd0d-fe716bb6aba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:58.149740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730a56c-bb85-4951-bd0d-fe716bb6aba8 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:58.150627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe9eb-5710-4f6c-9168-03210dfbff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:58.153833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730a56c-bb85-4951-bd0d-fe716bb6aba8 None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:58.153933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe9eb-5710-4f6c-9168-03210dfbff5a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:58.153933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe9eb-5710-4f6c-9168-03210dfbff5a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:58.153933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe9eb-5710-4f6c-9168-03210dfbff5a None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:58.154219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730a56c-bb85-4951-bd0d-fe716bb6aba8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:58.154303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fe9eb-5710-4f6c-9168-03210dfbff5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:58.154303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fe9eb-5710-4f6c-9168-03210dfbff5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:58.154427 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 230/442] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:57 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:58.154799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fe9eb-5710-4f6c-9168-03210dfbff5a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:58.154977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730a56c-bb85-4951-bd0d-fe716bb6aba8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:58.164596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 213/443] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:47:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:58.165395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe9eb-5710-4f6c-9168-03210dfbff5a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:58.256213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730a56c-bb85-4951-bd0d-fe716bb6aba8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:58.262913 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe9eb-5710-4f6c-9168-03210dfbff5a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:58.289036 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9d509-acb4-489b-8b0a-7928510a0c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:58.289825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9d509-acb4-489b-8b0a-7928510a0c5f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:58.290369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9d509-acb4-489b-8b0a-7928510a0c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:58.290473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9d509-acb4-489b-8b0a-7928510a0c5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:58.297637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6765c3-297c-4a3a-8ff8-db54f483c28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:58.298511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6765c3-297c-4a3a-8ff8-db54f483c28a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:58.298796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6765c3-297c-4a3a-8ff8-db54f483c28a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:58.299072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6765c3-297c-4a3a-8ff8-db54f483c28a None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:58.300904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6765c3-297c-4a3a-8ff8-db54f483c28a None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:58.301395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9d509-acb4-489b-8b0a-7928510a0c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:47:58.302363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6765c3-297c-4a3a-8ff8-db54f483c28a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:58.303958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 214/444] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:58 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:58.304709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6765c3-297c-4a3a-8ff8-db54f483c28a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-06f9d509-acb4-489b-8b0a-7928510a0c5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:47:58.309062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:47:58.310106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:47:58.310939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 231/445] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:47:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:47:58.312032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9d509-acb4-489b-8b0a-7928510a0c5f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:58.344041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:58.368578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:58.410520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9d509-acb4-489b-8b0a-7928510a0c5f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:58.411027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6765c3-297c-4a3a-8ff8-db54f483c28a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:58.444109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58c9bd-8f52-4d9d-9cf3-03a0961a8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:58.444989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58c9bd-8f52-4d9d-9cf3-03a0961a8c5c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:58.445270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58c9bd-8f52-4d9d-9cf3-03a0961a8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:58.445547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58c9bd-8f52-4d9d-9cf3-03a0961a8c5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:58.445645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e7f96-5137-4b86-8f41-8b12ed0b6f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:58.446221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e7f96-5137-4b86-8f41-8b12ed0b6f79 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:58.446406 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58c9bd-8f52-4d9d-9cf3-03a0961a8c5c None admin] RBAC: Adding query filter params (group.id=b0fd84213ca0455aaf2983a2b994abb9, role.id=a70b63ede7014fa9bbcb4e03275a2664, scope.project.id=35d52ce6b9e5491abc31216be989797e) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:58.446507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e7f96-5137-4b86-8f41-8b12ed0b6f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:58.446792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58c9bd-8f52-4d9d-9cf3-03a0961a8c5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:58.447054 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e7f96-5137-4b86-8f41-8b12ed0b6f79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:58.448248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58c9bd-8f52-4d9d-9cf3-03a0961a8c5c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:58.453927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 232/446] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 13 08:47:58 2022] GET /identity/v3/role_assignments?group.id=b0fd84213ca0455aaf2983a2b994abb9&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=a70b63ede7014fa9bbcb4e03275a2664 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:58.454725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58c9bd-8f52-4d9d-9cf3-03a0961a8c5c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:58.535056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e7f96-5137-4b86-8f41-8b12ed0b6f79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:58.536588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e7f96-5137-4b86-8f41-8b12ed0b6f79 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:58.568120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58c9bd-8f52-4d9d-9cf3-03a0961a8c5c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:58.568405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:58.592547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:58.605126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73907a42-7772-4c03-b731-73499ee74d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:58.605797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73907a42-7772-4c03-b731-73499ee74d44 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:58.606112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73907a42-7772-4c03-b731-73499ee74d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:58.606394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73907a42-7772-4c03-b731-73499ee74d44 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:58.607146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73907a42-7772-4c03-b731-73499ee74d44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:58.608156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73907a42-7772-4c03-b731-73499ee74d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:58.608860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73907a42-7772-4c03-b731-73499ee74d44 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:58.619348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 233/447] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:47:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:58.624882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73907a42-7772-4c03-b731-73499ee74d44 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:58.662219 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 215/448] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:47:58.667435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e7f96-5137-4b86-8f41-8b12ed0b6f79 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:58.745812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73907a42-7772-4c03-b731-73499ee74d44 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:58.785092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e7f96-5137-4b86-8f41-8b12ed0b6f79 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:58.786634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dbf38-ff73-439c-94f4-d8b1b3abecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:58.787285 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dbf38-ff73-439c-94f4-d8b1b3abecd5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:58.787697 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dbf38-ff73-439c-94f4-d8b1b3abecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:58.787862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dbf38-ff73-439c-94f4-d8b1b3abecd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:58.816279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:58.825169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d907614-da15-4ebc-8a15-991defec8312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:58.826053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d907614-da15-4ebc-8a15-991defec8312 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:58.826346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d907614-da15-4ebc-8a15-991defec8312 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:58.826648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d907614-da15-4ebc-8a15-991defec8312 None admin] PATH_INFO: `/v3/projects/35d52ce6b9e5491abc31216be989797e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:58.827829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d907614-da15-4ebc-8a15-991defec8312 None admin] RBAC: Authorizing `identity:get_project(project_id=35d52ce6b9e5491abc31216be989797e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:58.829253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d907614-da15-4ebc-8a15-991defec8312 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:58.831844 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 216/449] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:58 2022] GET /identity/v3/projects/35d52ce6b9e5491abc31216be989797e => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:58.837143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d907614-da15-4ebc-8a15-991defec8312 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:58.846468 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:58.861722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dbf38-ff73-439c-94f4-d8b1b3abecd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:58.863317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dbf38-ff73-439c-94f4-d8b1b3abecd5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:58.938472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d907614-da15-4ebc-8a15-991defec8312 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:58.962659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 234/450] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:47:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 08:47:58.968071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dbf38-ff73-439c-94f4-d8b1b3abecd5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:58.971933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fd5cf-9092-4d01-a11c-9669e85b8ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:58.972800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fd5cf-9092-4d01-a11c-9669e85b8ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:58.973077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fd5cf-9092-4d01-a11c-9669e85b8ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:58.973349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fd5cf-9092-4d01-a11c-9669e85b8ed1 None admin] PATH_INFO: `/v3/groups/b0fd84213ca0455aaf2983a2b994abb9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:58.974529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fd5cf-9092-4d01-a11c-9669e85b8ed1 None admin] RBAC: Authorizing `identity:get_group(group_id=b0fd84213ca0455aaf2983a2b994abb9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:58.975941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fd5cf-9092-4d01-a11c-9669e85b8ed1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:58.977550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 217/451] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:58 2022] GET /identity/v3/groups/b0fd84213ca0455aaf2983a2b994abb9 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:58.982972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fd5cf-9092-4d01-a11c-9669e85b8ed1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:59.042442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:59.052949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:59.068542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dbf38-ff73-439c-94f4-d8b1b3abecd5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:59.080805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fd5cf-9092-4d01-a11c-9669e85b8ed1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:59.100961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa24e9-7c5f-4df4-96b3-21f35963e9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:59.101812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa24e9-7c5f-4df4-96b3-21f35963e9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:59.102126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa24e9-7c5f-4df4-96b3-21f35963e9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:59.102404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa24e9-7c5f-4df4-96b3-21f35963e9d5 None admin] PATH_INFO: `/v3/projects/c8052949c90742f6ada3f8a34d5636f8` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:59.103581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa24e9-7c5f-4df4-96b3-21f35963e9d5 None admin] RBAC: Authorizing `identity:get_project(project_id=c8052949c90742f6ada3f8a34d5636f8)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:59.105023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa24e9-7c5f-4df4-96b3-21f35963e9d5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:59.111128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 235/452] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:47:58 2022] GET /identity/v3/projects/c8052949c90742f6ada3f8a34d5636f8 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:59.112618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa24e9-7c5f-4df4-96b3-21f35963e9d5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:59.116615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83908629-3cb1-4dfe-abf6-73eb7edefca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:59.117250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83908629-3cb1-4dfe-abf6-73eb7edefca9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:59.117529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83908629-3cb1-4dfe-abf6-73eb7edefca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:59.117802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83908629-3cb1-4dfe-abf6-73eb7edefca9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:59.118655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83908629-3cb1-4dfe-abf6-73eb7edefca9 None admin] RBAC: Adding query filter params (group.id=b0fd84213ca0455aaf2983a2b994abb9, role.id=120eeffad7094e10a69166d19aaa6439, scope.project.id=35d52ce6b9e5491abc31216be989797e) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:59.118966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83908629-3cb1-4dfe-abf6-73eb7edefca9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:59.120389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83908629-3cb1-4dfe-abf6-73eb7edefca9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:59.126612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 218/453] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 13 08:47:58 2022] GET /identity/v3/role_assignments?group.id=b0fd84213ca0455aaf2983a2b994abb9&scope.project.id=35d52ce6b9e5491abc31216be989797e&role.id=120eeffad7094e10a69166d19aaa6439 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:47:59.224666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa24e9-7c5f-4df4-96b3-21f35963e9d5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:59.265161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53beb95-8d18-4ca5-aab0-2026693ad433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:59.266061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53beb95-8d18-4ca5-aab0-2026693ad433 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:59.266365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53beb95-8d18-4ca5-aab0-2026693ad433 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:59.266641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53beb95-8d18-4ca5-aab0-2026693ad433 None admin] PATH_INFO: `/v3/groups/53768a0eacd3460c8f3df0771a018665` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:59.267862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53beb95-8d18-4ca5-aab0-2026693ad433 None admin] RBAC: Authorizing `identity:get_group(group_id=53768a0eacd3460c8f3df0771a018665)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:59.269672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53beb95-8d18-4ca5-aab0-2026693ad433 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:59.274081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 236/454] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 13 08:47:59 2022] GET /identity/v3/groups/53768a0eacd3460c8f3df0771a018665 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:47:59.277636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83908629-3cb1-4dfe-abf6-73eb7edefca9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:47:59.406538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83908629-3cb1-4dfe-abf6-73eb7edefca9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:47:59.442686 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:47:59.454117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387361bd-daca-4bb1-ba2b-e02a874ef307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:47:59.454859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387361bd-daca-4bb1-ba2b-e02a874ef307 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:47:59.455132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387361bd-daca-4bb1-ba2b-e02a874ef307 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:47:59.455402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387361bd-daca-4bb1-ba2b-e02a874ef307 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:47:59.456832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387361bd-daca-4bb1-ba2b-e02a874ef307 None admin] RBAC: Adding query filter params (group.id=53768a0eacd3460c8f3df0771a018665, role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=c8052949c90742f6ada3f8a34d5636f8) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:47:59.464352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387361bd-daca-4bb1-ba2b-e02a874ef307 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:47:59.464352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387361bd-daca-4bb1-ba2b-e02a874ef307 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:47:59.464576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 219/455] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 13 08:47:59 2022] GET /identity/v3/role_assignments?group.id=53768a0eacd3460c8f3df0771a018665&scope.project.id=c8052949c90742f6ada3f8a34d5636f8&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:02.029345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2c811-e6ad-48d4-bb3b-ce1418f2bb1d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.030246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2c811-e6ad-48d4-bb3b-ce1418f2bb1d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.030663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2c811-e6ad-48d4-bb3b-ce1418f2bb1d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.032673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 237/456] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:02.038164 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7882ebe-3c07-4881-9804-f6aec24194ff None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.038895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7882ebe-3c07-4881-9804-f6aec24194ff None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.039329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7882ebe-3c07-4881-9804-f6aec24194ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.062052 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f258bed-43ea-4f12-b818-06b2b697c6f0 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.062343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f258bed-43ea-4f12-b818-06b2b697c6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.062629 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f258bed-43ea-4f12-b818-06b2b697c6f0 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.063553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a7882ebe-3c07-4881-9804-f6aec24194ff None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:02.064219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 238/457] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:02.064778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a7882ebe-3c07-4881-9804-f6aec24194ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:02.069609 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca32e6-5ad0-4a1f-ae6b-c1188cd7735e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.069895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca32e6-5ad0-4a1f-ae6b-c1188cd7735e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.070204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca32e6-5ad0-4a1f-ae6b-c1188cd7735e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.099347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-baca32e6-5ad0-4a1f-ae6b-c1188cd7735e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:02.100569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-baca32e6-5ad0-4a1f-ae6b-c1188cd7735e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:02.134963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:02.208578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 220/458] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:02.232463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20036b-fbd7-4cb9-a9bf-c25e8a720db3 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.232820 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20036b-fbd7-4cb9-a9bf-c25e8a720db3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.233130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20036b-fbd7-4cb9-a9bf-c25e8a720db3 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.234173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 239/459] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:02.235061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca32e6-5ad0-4a1f-ae6b-c1188cd7735e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:02.235217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 221/460] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:02.244591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb012c-ec71-4f00-8b0a-5c8856590b26 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.245131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb012c-ec71-4f00-8b0a-5c8856590b26 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.245559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb012c-ec71-4f00-8b0a-5c8856590b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.275602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6cdb012c-ec71-4f00-8b0a-5c8856590b26 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:02.277403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6cdb012c-ec71-4f00-8b0a-5c8856590b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:02.366136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 222/461] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:48:02.371059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb012c-ec71-4f00-8b0a-5c8856590b26 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:02.385066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca32e6-5ad0-4a1f-ae6b-c1188cd7735e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:02.426947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baa894-1d42-4a28-94f5-2781110c1b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:02.427754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baa894-1d42-4a28-94f5-2781110c1b75 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.428041 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baa894-1d42-4a28-94f5-2781110c1b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.428318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baa894-1d42-4a28-94f5-2781110c1b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.443581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baa894-1d42-4a28-94f5-2781110c1b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:02.445061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baa894-1d42-4a28-94f5-2781110c1b75 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:02.459903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-24baa894-1d42-4a28-94f5-2781110c1b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:02.469796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:02.473238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:02.475081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:02.475081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:02.475081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:02.475081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:02.476685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 240/462] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:02.482470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baa894-1d42-4a28-94f5-2781110c1b75 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:02.486066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:02.524037 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb012c-ec71-4f00-8b0a-5c8856590b26 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:02.567891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebc27c-ce00-4db1-b511-ccc557743864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:02.568948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebc27c-ce00-4db1-b511-ccc557743864 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.569372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebc27c-ce00-4db1-b511-ccc557743864 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.569788 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebc27c-ce00-4db1-b511-ccc557743864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.585495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebc27c-ce00-4db1-b511-ccc557743864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:02.587230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebc27c-ce00-4db1-b511-ccc557743864 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:02.607134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baa894-1d42-4a28-94f5-2781110c1b75 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-93ebc27c-ce00-4db1-b511-ccc557743864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:02.607582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:02.608672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:02.609647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:02.609647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:02.609647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:02.609647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:02.611503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 223/463] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:02.613124 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebc27c-ce00-4db1-b511-ccc557743864 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:02.649176 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807f130-c1f3-404b-958e-3833cbf8d66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:02.649973 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807f130-c1f3-404b-958e-3833cbf8d66a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.650286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807f130-c1f3-404b-958e-3833cbf8d66a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.650564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807f130-c1f3-404b-958e-3833cbf8d66a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.662651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807f130-c1f3-404b-958e-3833cbf8d66a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:02.664191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807f130-c1f3-404b-958e-3833cbf8d66a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7807f130-c1f3-404b-958e-3833cbf8d66a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:02.683221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:02.684290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:02.685158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:02.685158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:02.685158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:02.685158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:02.689283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 241/464] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:02.690210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807f130-c1f3-404b-958e-3833cbf8d66a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:02.741764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebc27c-ce00-4db1-b511-ccc557743864 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:02.743621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:02.751036 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:02.780676 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65449241-fee6-425d-a674-c7f061a42d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:02.781664 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65449241-fee6-425d-a674-c7f061a42d8f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.782236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65449241-fee6-425d-a674-c7f061a42d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.782915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65449241-fee6-425d-a674-c7f061a42d8f None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.784245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65449241-fee6-425d-a674-c7f061a42d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:02.784819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65449241-fee6-425d-a674-c7f061a42d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:02.786391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65449241-fee6-425d-a674-c7f061a42d8f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:02.800207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807f130-c1f3-404b-958e-3833cbf8d66a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:02.802561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 224/465] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:02.804314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65449241-fee6-425d-a674-c7f061a42d8f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:02.838941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112f3b6-66b7-4a85-aaf1-3acee9ded11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:02.839606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112f3b6-66b7-4a85-aaf1-3acee9ded11a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.839898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112f3b6-66b7-4a85-aaf1-3acee9ded11a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.840356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112f3b6-66b7-4a85-aaf1-3acee9ded11a None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.840970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112f3b6-66b7-4a85-aaf1-3acee9ded11a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:02.841289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112f3b6-66b7-4a85-aaf1-3acee9ded11a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:02.842708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112f3b6-66b7-4a85-aaf1-3acee9ded11a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:02.867065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 242/466] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:02.867927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112f3b6-66b7-4a85-aaf1-3acee9ded11a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:02.927160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65449241-fee6-425d-a674-c7f061a42d8f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:02.968459 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ddd37-25dc-4341-af6b-2d5b5f0d3d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:02.969423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddd37-25dc-4341-af6b-2d5b5f0d3d9d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:02.969887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddd37-25dc-4341-af6b-2d5b5f0d3d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:02.970545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddd37-25dc-4341-af6b-2d5b5f0d3d9d None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:02.972592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ddd37-25dc-4341-af6b-2d5b5f0d3d9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:02.973065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ddd37-25dc-4341-af6b-2d5b5f0d3d9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:02.974645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ddd37-25dc-4341-af6b-2d5b5f0d3d9d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:02.980659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112f3b6-66b7-4a85-aaf1-3acee9ded11a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:02.991833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 225/467] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:02.992670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ddd37-25dc-4341-af6b-2d5b5f0d3d9d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:02.999705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:03.019160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:03.026056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e445a-745c-4fe4-9e9d-d055fd0f67b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:03.026730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e445a-745c-4fe4-9e9d-d055fd0f67b0 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:03.027011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e445a-745c-4fe4-9e9d-d055fd0f67b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:03.027498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e445a-745c-4fe4-9e9d-d055fd0f67b0 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:03.028034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e445a-745c-4fe4-9e9d-d055fd0f67b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:03.029561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e445a-745c-4fe4-9e9d-d055fd0f67b0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:03.099612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ddd37-25dc-4341-af6b-2d5b5f0d3d9d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:03.134160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 243/468] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:48:02 2022] POST /identity/v3/users => generated 248 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:03.134160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0e9cb-fc74-4fea-9e56-4f5381f1f9fd None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:03.134160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0e9cb-fc74-4fea-9e56-4f5381f1f9fd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:03.134160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0e9cb-fc74-4fea-9e56-4f5381f1f9fd None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:03.134160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 244/469] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:03.134160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0e9cb-fc74-4fea-9e56-4f5381f1f9fd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:03.146043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dd40d-702b-447e-ac42-90ce6414db0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:03.146833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dd40d-702b-447e-ac42-90ce6414db0f None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:03.147122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dd40d-702b-447e-ac42-90ce6414db0f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:03.147461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dd40d-702b-447e-ac42-90ce6414db0f None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:03.148319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dd40d-702b-447e-ac42-90ce6414db0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:03.149916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dd40d-702b-447e-ac42-90ce6414db0f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:03.242650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 226/470] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:48:02 2022] POST /identity/v3/users => generated 246 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:03.244719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e78c3-a8b2-4b88-ab3a-3e8abef6d535 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:03.244843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e78c3-a8b2-4b88-ab3a-3e8abef6d535 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:03.245082 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e78c3-a8b2-4b88-ab3a-3e8abef6d535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:03.262370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0e9cb-fc74-4fea-9e56-4f5381f1f9fd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:03.287215 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4c3e78c3-a8b2-4b88-ab3a-3e8abef6d535 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:03.288571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4c3e78c3-a8b2-4b88-ab3a-3e8abef6d535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:03.312047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50b586-cd6f-4bad-95e9-a6d35fdc6785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:03.312730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50b586-cd6f-4bad-95e9-a6d35fdc6785 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:03.313011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50b586-cd6f-4bad-95e9-a6d35fdc6785 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:03.313284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50b586-cd6f-4bad-95e9-a6d35fdc6785 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:03.313998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50b586-cd6f-4bad-95e9-a6d35fdc6785 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:03.317855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50b586-cd6f-4bad-95e9-a6d35fdc6785 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:03.400306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 227/471] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:03.401961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e39014-17c6-479e-b629-2140e7b2487c None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:03.402473 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e39014-17c6-479e-b629-2140e7b2487c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:03.402946 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e39014-17c6-479e-b629-2140e7b2487c None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:03.404446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 228/472] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:03.410321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343d147-5092-423b-abaa-c92a952a7af1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:03.410654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343d147-5092-423b-abaa-c92a952a7af1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:03.410935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343d147-5092-423b-abaa-c92a952a7af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:03.429336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0343d147-5092-423b-abaa-c92a952a7af1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:03.430583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0343d147-5092-423b-abaa-c92a952a7af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:03.432951 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:03.444064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 245/473] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:48:03 2022] POST /identity/v3/users => generated 248 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:03.444924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50b586-cd6f-4bad-95e9-a6d35fdc6785 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:03.490516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:03.519964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 229/474] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:48:03.545974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343d147-5092-423b-abaa-c92a952a7af1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:03.601368 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50b586-cd6f-4bad-95e9-a6d35fdc6785 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:03.662915 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1df9b-8d0d-43a0-9db8-6927a8a10aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:03.663955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1df9b-8d0d-43a0-9db8-6927a8a10aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:03.666990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1df9b-8d0d-43a0-9db8-6927a8a10aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:03.667437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1df9b-8d0d-43a0-9db8-6927a8a10aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:03.681156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1df9b-8d0d-43a0-9db8-6927a8a10aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:03.682308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343d147-5092-423b-abaa-c92a952a7af1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:03.683103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1df9b-8d0d-43a0-9db8-6927a8a10aa5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-62a1df9b-8d0d-43a0-9db8-6927a8a10aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:03.703092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:03.704171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:03.705206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:03.705206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:03.705206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:03.705206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:03.707154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 246/475] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:03.713381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1df9b-8d0d-43a0-9db8-6927a8a10aa5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:03.732949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba320584-28bb-460c-9c71-d72810b9c6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:03.733574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba320584-28bb-460c-9c71-d72810b9c6b8 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:03.733860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba320584-28bb-460c-9c71-d72810b9c6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:03.734171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba320584-28bb-460c-9c71-d72810b9c6b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:03.734929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba320584-28bb-460c-9c71-d72810b9c6b8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:03.736321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba320584-28bb-460c-9c71-d72810b9c6b8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:03.771258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.sql.core [None req-ba320584-28bb-460c-9c71-d72810b9c6b8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 08:48:03.771258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 13 08:48:03.771258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [parameters: {'id': '2c7a69016a4c4fbfad89278b6cae8e5e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 13 08:48:03.771258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80288) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ba320584-28bb-460c-9c71-d72810b9c6b8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application conn.query(q) Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application result.read() Jul 13 08:48:03.787763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return ref.to_dict() Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 08:48:03.788703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application next(self.gen) Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application yield resource Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application next(self.gen) Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise self.value Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application session.commit() Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 08:48:03.790139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self.session.flush() Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application compat.raise_( Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 08:48:03.790982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 08:48:03.791865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application conn.query(q) Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application result.read() Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 08:48:03.794582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [parameters: {'id': '2c7a69016a4c4fbfad89278b6cae8e5e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:03.797153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 13 08:48:03.800273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:03.801891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 230/476] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 13 08:48:03 2022] POST /identity/v3/roles => generated 144 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:03.807998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba320584-28bb-460c-9c71-d72810b9c6b8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:03.812171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:03.841854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:03.867536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1df9b-8d0d-43a0-9db8-6927a8a10aa5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:03.905852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162eb2e-85f9-485b-9f71-f3c3f59fc9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:03.906961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162eb2e-85f9-485b-9f71-f3c3f59fc9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:03.907640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162eb2e-85f9-485b-9f71-f3c3f59fc9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:03.908118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162eb2e-85f9-485b-9f71-f3c3f59fc9f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:03.909601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162eb2e-85f9-485b-9f71-f3c3f59fc9f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:03.910155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162eb2e-85f9-485b-9f71-f3c3f59fc9f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:03.911783 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162eb2e-85f9-485b-9f71-f3c3f59fc9f5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:03.926886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 247/477] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:03.932589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162eb2e-85f9-485b-9f71-f3c3f59fc9f5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:03.951243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba320584-28bb-460c-9c71-d72810b9c6b8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:04.012791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbb7ab-63f8-40c3-b207-a03bae39dfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:04.014094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbb7ab-63f8-40c3-b207-a03bae39dfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:04.014383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbb7ab-63f8-40c3-b207-a03bae39dfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:04.014658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbb7ab-63f8-40c3-b207-a03bae39dfb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:04.039289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbb7ab-63f8-40c3-b207-a03bae39dfb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b3fbb7ab-63f8-40c3-b207-a03bae39dfb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:04.055730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:04.057432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 08:48:04.058261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:04.059262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 231/478] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:48:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:04.067576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbb7ab-63f8-40c3-b207-a03bae39dfb1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:04.126692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162eb2e-85f9-485b-9f71-f3c3f59fc9f5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:04.198545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:04.222579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06965c8d-15d8-4f24-9779-d88972a43dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:04.223331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06965c8d-15d8-4f24-9779-d88972a43dfa None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:04.223628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06965c8d-15d8-4f24-9779-d88972a43dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:04.223902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06965c8d-15d8-4f24-9779-d88972a43dfa None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:04.225803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06965c8d-15d8-4f24-9779-d88972a43dfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:04.228840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06965c8d-15d8-4f24-9779-d88972a43dfa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:04.244277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbb7ab-63f8-40c3-b207-a03bae39dfb1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:04.246984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:04.285701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e09456-7353-462b-b571-0e581673a810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:04.286638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e09456-7353-462b-b571-0e581673a810 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:04.286969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e09456-7353-462b-b571-0e581673a810 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:04.287304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e09456-7353-462b-b571-0e581673a810 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:04.288176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e09456-7353-462b-b571-0e581673a810 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:04.288503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e09456-7353-462b-b571-0e581673a810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:04.289874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e09456-7353-462b-b571-0e581673a810 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:04.299953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 232/479] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:04.344491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 248/480] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:48:03 2022] POST /identity/v3/users => generated 249 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:05.587871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a825c9f-f481-4f8c-a05a-2d68c4cd825d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:05.588913 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a825c9f-f481-4f8c-a05a-2d68c4cd825d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:05.589409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a825c9f-f481-4f8c-a05a-2d68c4cd825d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:05.591250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 233/481] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:05.596162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c44067-5b17-48c3-beac-c32b4a3c416a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:05.597004 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c44067-5b17-48c3-beac-c32b4a3c416a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:05.597473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c44067-5b17-48c3-beac-c32b4a3c416a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:05.619485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-34c44067-5b17-48c3-beac-c32b4a3c416a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:05.620989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-34c44067-5b17-48c3-beac-c32b4a3c416a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:05.693202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd613c-728b-469b-8c20-d71ff75bc8ea None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:05.693524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd613c-728b-469b-8c20-d71ff75bc8ea None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:05.693759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd613c-728b-469b-8c20-d71ff75bc8ea None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:05.695290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 234/482] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:05.701362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366dcc0-9bd0-4909-9c2c-6be4a7a1e269 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:05.701886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366dcc0-9bd0-4909-9c2c-6be4a7a1e269 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:05.702412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366dcc0-9bd0-4909-9c2c-6be4a7a1e269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:05.708700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 249/483] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:48:05.720041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0366dcc0-9bd0-4909-9c2c-6be4a7a1e269 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:05.722350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0366dcc0-9bd0-4909-9c2c-6be4a7a1e269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:05.745379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c44067-5b17-48c3-beac-c32b4a3c416a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:05.812732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 235/484] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:48:05.849158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366dcc0-9bd0-4909-9c2c-6be4a7a1e269 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:05.934239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c44067-5b17-48c3-beac-c32b4a3c416a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:05.946415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:05.976786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:06.000700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bd234-c306-4db9-9da8-e9ba8708ae8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:06.001494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bd234-c306-4db9-9da8-e9ba8708ae8b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.001773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bd234-c306-4db9-9da8-e9ba8708ae8b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.002061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bd234-c306-4db9-9da8-e9ba8708ae8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.019822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bd234-c306-4db9-9da8-e9ba8708ae8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6c7bd234-c306-4db9-9da8-e9ba8708ae8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:06.027591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:06.028497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:06.029289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 250/485] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:06.034221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366dcc0-9bd0-4909-9c2c-6be4a7a1e269 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:06.036553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bd234-c306-4db9-9da8-e9ba8708ae8b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:06.089962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad19611-9b65-4450-96d3-58cb8b91cf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:06.092597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad19611-9b65-4450-96d3-58cb8b91cf5d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.092597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad19611-9b65-4450-96d3-58cb8b91cf5d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.092597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad19611-9b65-4450-96d3-58cb8b91cf5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.104534 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad19611-9b65-4450-96d3-58cb8b91cf5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-bad19611-9b65-4450-96d3-58cb8b91cf5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:06.112702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:06.113648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:06.114622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 236/486] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:06.115463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960f7c1-31b6-4402-802f-2de21d746aaa None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.115463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960f7c1-31b6-4402-802f-2de21d746aaa None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.121639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960f7c1-31b6-4402-802f-2de21d746aaa None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.121639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 237/487] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:06.122539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960f7c1-31b6-4402-802f-2de21d746aaa None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:06.210704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bd234-c306-4db9-9da8-e9ba8708ae8b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:06.247977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2a2e4-3dc1-4d05-b5d7-0841c822210a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:06.248736 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2a2e4-3dc1-4d05-b5d7-0841c822210a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.249028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2a2e4-3dc1-4d05-b5d7-0841c822210a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.249304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2a2e4-3dc1-4d05-b5d7-0841c822210a None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.250062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2a2e4-3dc1-4d05-b5d7-0841c822210a None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:06.250402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2a2e4-3dc1-4d05-b5d7-0841c822210a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:06.251993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2a2e4-3dc1-4d05-b5d7-0841c822210a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:06.256408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960f7c1-31b6-4402-802f-2de21d746aaa None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:06.262145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 251/488] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:06.263414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e3927-ffbd-43aa-a5e8-3f631f728aa8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.263706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e3927-ffbd-43aa-a5e8-3f631f728aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.263982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e3927-ffbd-43aa-a5e8-3f631f728aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.273935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:06.288180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-081e3927-ffbd-43aa-a5e8-3f631f728aa8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:06.290091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-081e3927-ffbd-43aa-a5e8-3f631f728aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:06.299207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59523f7-b0ad-4484-84e8-bbb4cda91777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:06.300369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59523f7-b0ad-4484-84e8-bbb4cda91777 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.300869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59523f7-b0ad-4484-84e8-bbb4cda91777 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.301337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59523f7-b0ad-4484-84e8-bbb4cda91777 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.302339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59523f7-b0ad-4484-84e8-bbb4cda91777 None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:06.302840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59523f7-b0ad-4484-84e8-bbb4cda91777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:06.304400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59523f7-b0ad-4484-84e8-bbb4cda91777 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:06.313777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 238/489] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:06.315624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59523f7-b0ad-4484-84e8-bbb4cda91777 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:06.337873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:06.364395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 252/490] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:06.364991 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e3927-ffbd-43aa-a5e8-3f631f728aa8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:06.433173 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59523f7-b0ad-4484-84e8-bbb4cda91777 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:06.469759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d3c0a-d11f-40e8-8724-0729195d94f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:06.470848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d3c0a-d11f-40e8-8724-0729195d94f3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.471785 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d3c0a-d11f-40e8-8724-0729195d94f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.472381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d3c0a-d11f-40e8-8724-0729195d94f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.472592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e3927-ffbd-43aa-a5e8-3f631f728aa8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:06.484031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d3c0a-d11f-40e8-8724-0729195d94f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:06.485718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d3c0a-d11f-40e8-8724-0729195d94f3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-2b8d3c0a-d11f-40e8-8724-0729195d94f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:06.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:06.506293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:06.507154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:06.507154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:06.507154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:06.507154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:06.509163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 239/491] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:06.510120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d3c0a-d11f-40e8-8724-0729195d94f3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:06.511053 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a30441-e6b0-4400-a748-1bc8151fcafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:06.511950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a30441-e6b0-4400-a748-1bc8151fcafe None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.512326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a30441-e6b0-4400-a748-1bc8151fcafe None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.512694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a30441-e6b0-4400-a748-1bc8151fcafe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.524251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a30441-e6b0-4400-a748-1bc8151fcafe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:06.525898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a30441-e6b0-4400-a748-1bc8151fcafe None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:06.542097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a6a30441-e6b0-4400-a748-1bc8151fcafe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:06.544980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:06.546110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:06.546977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:06.546977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:06.546977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:06.546977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:06.546977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 253/492] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:06.548047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a30441-e6b0-4400-a748-1bc8151fcafe None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:06.580630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:06.622694 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d3c0a-d11f-40e8-8724-0729195d94f3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:06.656696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a30441-e6b0-4400-a748-1bc8151fcafe None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:06.661446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76b087-c5e6-4dd3-9cf3-9592d7b4f46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:06.662595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76b087-c5e6-4dd3-9cf3-9592d7b4f46b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.663057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76b087-c5e6-4dd3-9cf3-9592d7b4f46b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.663543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76b087-c5e6-4dd3-9cf3-9592d7b4f46b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.676133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76b087-c5e6-4dd3-9cf3-9592d7b4f46b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-1f76b087-c5e6-4dd3-9cf3-9592d7b4f46b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:06.684322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:06.685199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:06.685991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:06.688683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 240/493] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:06.689634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76b087-c5e6-4dd3-9cf3-9592d7b4f46b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:06.695394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c96fcf-5b96-4705-b969-9f6b2e526e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:06.696147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c96fcf-5b96-4705-b969-9f6b2e526e87 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.696490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c96fcf-5b96-4705-b969-9f6b2e526e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.696847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c96fcf-5b96-4705-b969-9f6b2e526e87 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.697634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c96fcf-5b96-4705-b969-9f6b2e526e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:06.698001 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c96fcf-5b96-4705-b969-9f6b2e526e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:06.699469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c96fcf-5b96-4705-b969-9f6b2e526e87 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:06.712547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 254/494] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:06.713559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c96fcf-5b96-4705-b969-9f6b2e526e87 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:06.785629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:06.804298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76b087-c5e6-4dd3-9cf3-9592d7b4f46b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:06.812968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:06.825641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c96fcf-5b96-4705-b969-9f6b2e526e87 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:06.857228 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587df65-40c2-4523-8f2d-0d6954b0a718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:06.857885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587df65-40c2-4523-8f2d-0d6954b0a718 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.858194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587df65-40c2-4523-8f2d-0d6954b0a718 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.858464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587df65-40c2-4523-8f2d-0d6954b0a718 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.859180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587df65-40c2-4523-8f2d-0d6954b0a718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:06.859485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587df65-40c2-4523-8f2d-0d6954b0a718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:06.860839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587df65-40c2-4523-8f2d-0d6954b0a718 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:06.871378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd4647-dbda-46bd-af0f-e3aec4a712f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:06.872489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd4647-dbda-46bd-af0f-e3aec4a712f1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:06.872990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd4647-dbda-46bd-af0f-e3aec4a712f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:06.873407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd4647-dbda-46bd-af0f-e3aec4a712f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:06.874316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 241/495] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:06.874572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd4647-dbda-46bd-af0f-e3aec4a712f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:06.875078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd4647-dbda-46bd-af0f-e3aec4a712f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:06.875409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587df65-40c2-4523-8f2d-0d6954b0a718 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:06.876912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd4647-dbda-46bd-af0f-e3aec4a712f1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:06.889053 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 255/496] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:06.890506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd4647-dbda-46bd-af0f-e3aec4a712f1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:06.983892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587df65-40c2-4523-8f2d-0d6954b0a718 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:07.006995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd4647-dbda-46bd-af0f-e3aec4a712f1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:07.021361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd397b-28d1-4064-8dbd-875d16e9817a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:07.022663 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd397b-28d1-4064-8dbd-875d16e9817a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.023056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd397b-28d1-4064-8dbd-875d16e9817a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.023334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd397b-28d1-4064-8dbd-875d16e9817a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.033233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd397b-28d1-4064-8dbd-875d16e9817a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:07.035905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd397b-28d1-4064-8dbd-875d16e9817a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:07.041232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0178c-82db-4917-a9a5-67edf424e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:07.042022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0178c-82db-4917-a9a5-67edf424e21d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.042332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0178c-82db-4917-a9a5-67edf424e21d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.042605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0178c-82db-4917-a9a5-67edf424e21d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.050853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0178c-82db-4917-a9a5-67edf424e21d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:07.052306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0178c-82db-4917-a9a5-67edf424e21d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-03bd397b-28d1-4064-8dbd-875d16e9817a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:07.053772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:07.062519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 242/497] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:07.063520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd397b-28d1-4064-8dbd-875d16e9817a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:07.068345 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-99f0178c-82db-4917-a9a5-67edf424e21d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:07.068703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:07.069600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:07.070541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:07.070541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:07.070541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:07.070541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:07.070541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:07.070541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:07.070541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:07.070541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:07.070541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:07.070541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:07.074434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 256/498] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 13 08:48:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:07.075731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9fcc1-3687-4bbc-8a07-7b6b547f8aac None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.076021 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9fcc1-3687-4bbc-8a07-7b6b547f8aac None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.076299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9fcc1-3687-4bbc-8a07-7b6b547f8aac None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.077586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 257/499] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:07.078730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aecca-6494-4f78-9de4-96dd802a223c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.079013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aecca-6494-4f78-9de4-96dd802a223c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.079286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aecca-6494-4f78-9de4-96dd802a223c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.081009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 258/500] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:07.081681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aecca-6494-4f78-9de4-96dd802a223c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:07.095017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:07.164574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd397b-28d1-4064-8dbd-875d16e9817a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:07.183170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aecca-6494-4f78-9de4-96dd802a223c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:07.197711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b521aa-86d6-4de9-a08c-24667edb74bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:07.198529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b521aa-86d6-4de9-a08c-24667edb74bd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.198809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b521aa-86d6-4de9-a08c-24667edb74bd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.199083 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b521aa-86d6-4de9-a08c-24667edb74bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.209855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b521aa-86d6-4de9-a08c-24667edb74bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:07.211361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b521aa-86d6-4de9-a08c-24667edb74bd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:07.216744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156c4a0-68e1-4b3f-9dae-183829ca1767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:07.217394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156c4a0-68e1-4b3f-9dae-183829ca1767 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.217674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156c4a0-68e1-4b3f-9dae-183829ca1767 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.217944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156c4a0-68e1-4b3f-9dae-183829ca1767 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.218809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156c4a0-68e1-4b3f-9dae-183829ca1767 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:07.219115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156c4a0-68e1-4b3f-9dae-183829ca1767 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:07.220548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156c4a0-68e1-4b3f-9dae-183829ca1767 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-76b521aa-86d6-4de9-a08c-24667edb74bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:07.228181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:07.229040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:07.229897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:07.229897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:07.229897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:07.229897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:07.229897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 243/501] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:07.230500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b521aa-86d6-4de9-a08c-24667edb74bd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:07.258078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 259/502] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 13 08:48:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:07.259400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee7f72-3cf5-4717-9e49-aff7cf68cad6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.259688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee7f72-3cf5-4717-9e49-aff7cf68cad6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.260074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee7f72-3cf5-4717-9e49-aff7cf68cad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.276877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a6ee7f72-3cf5-4717-9e49-aff7cf68cad6 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:07.278108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a6ee7f72-3cf5-4717-9e49-aff7cf68cad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:07.296282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:07.338188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:07.341224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b521aa-86d6-4de9-a08c-24667edb74bd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:07.354152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 260/503] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:07.354152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f03c9-da49-40f7-a901-c596f4a55582 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.354152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f03c9-da49-40f7-a901-c596f4a55582 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.354152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f03c9-da49-40f7-a901-c596f4a55582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.362905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-736f03c9-da49-40f7-a901-c596f4a55582 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:07.364083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-736f03c9-da49-40f7-a901-c596f4a55582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:07.377082 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf075d2-892e-4ff6-b37c-7f4b7922aee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:07.378427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf075d2-892e-4ff6-b37c-7f4b7922aee0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.378706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf075d2-892e-4ff6-b37c-7f4b7922aee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.378975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf075d2-892e-4ff6-b37c-7f4b7922aee0 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.379815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf075d2-892e-4ff6-b37c-7f4b7922aee0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:07.380123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf075d2-892e-4ff6-b37c-7f4b7922aee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:07.381546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf075d2-892e-4ff6-b37c-7f4b7922aee0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:07.418985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 244/504] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 13 08:48:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:07.419580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf075d2-892e-4ff6-b37c-7f4b7922aee0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:07.437341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 261/505] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:07.438200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f03c9-da49-40f7-a901-c596f4a55582 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:07.518528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf075d2-892e-4ff6-b37c-7f4b7922aee0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:07.535535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:07.536866 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f03c9-da49-40f7-a901-c596f4a55582 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:07.552786 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b209f40-250c-415c-9bf9-004e7ef0b9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:07.553439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b209f40-250c-415c-9bf9-004e7ef0b9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.553722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b209f40-250c-415c-9bf9-004e7ef0b9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.553994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b209f40-250c-415c-9bf9-004e7ef0b9a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.554750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b209f40-250c-415c-9bf9-004e7ef0b9a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:07.555079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b209f40-250c-415c-9bf9-004e7ef0b9a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:07.556446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b209f40-250c-415c-9bf9-004e7ef0b9a0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:07.563847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:07.568946 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 245/506] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:07.569886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b209f40-250c-415c-9bf9-004e7ef0b9a0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:07.573457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf520fc-5a37-40bf-9bcd-a6ba93e8b4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:07.574136 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf520fc-5a37-40bf-9bcd-a6ba93e8b4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.574421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf520fc-5a37-40bf-9bcd-a6ba93e8b4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.574695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf520fc-5a37-40bf-9bcd-a6ba93e8b4d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.641588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf520fc-5a37-40bf-9bcd-a6ba93e8b4d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:07.643183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf520fc-5a37-40bf-9bcd-a6ba93e8b4d3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:07.671629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b209f40-250c-415c-9bf9-004e7ef0b9a0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:07.704699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe58d94-6213-4c13-8a5f-4d810516a635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:07.705480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe58d94-6213-4c13-8a5f-4d810516a635 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.705748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe58d94-6213-4c13-8a5f-4d810516a635 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.706020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe58d94-6213-4c13-8a5f-4d810516a635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.715707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe58d94-6213-4c13-8a5f-4d810516a635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:07.717132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe58d94-6213-4c13-8a5f-4d810516a635 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ebe58d94-6213-4c13-8a5f-4d810516a635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:07.734491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:07.735315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:07.736195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:07.736195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:07.736195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:07.736195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:07.736195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 246/507] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:07.736453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe58d94-6213-4c13-8a5f-4d810516a635 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:07.745139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 262/508] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:07.746284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf520fc-5a37-40bf-9bcd-a6ba93e8b4d3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:07.768828 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:07.775018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:07.837380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe58d94-6213-4c13-8a5f-4d810516a635 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:07.844922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf520fc-5a37-40bf-9bcd-a6ba93e8b4d3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:07.870367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c60ce-3ab3-4932-b90a-e6588e2abab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:07.871005 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c60ce-3ab3-4932-b90a-e6588e2abab4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.871284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c60ce-3ab3-4932-b90a-e6588e2abab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.871561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c60ce-3ab3-4932-b90a-e6588e2abab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.878266 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d542b-0a35-4080-b061-2a9c04b8a2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:07.879040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d542b-0a35-4080-b061-2a9c04b8a2bc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:07.879316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d542b-0a35-4080-b061-2a9c04b8a2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:07.879585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d542b-0a35-4080-b061-2a9c04b8a2bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:07.889417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d542b-0a35-4080-b061-2a9c04b8a2bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-732d542b-0a35-4080-b061-2a9c04b8a2bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:07.897657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:07.898570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:07.899360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 263/509] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:07.900010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d542b-0a35-4080-b061-2a9c04b8a2bc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:07.943935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c60ce-3ab3-4932-b90a-e6588e2abab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:07.945334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c60ce-3ab3-4932-b90a-e6588e2abab4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:07.974216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:07.988254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:08.000706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d542b-0a35-4080-b061-2a9c04b8a2bc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.036809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befb474-a184-4af8-a051-6a0aa6b9e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.037760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befb474-a184-4af8-a051-6a0aa6b9e8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.038517 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befb474-a184-4af8-a051-6a0aa6b9e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.038684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befb474-a184-4af8-a051-6a0aa6b9e8e4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:08.042576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 247/510] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:08.043392 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c60ce-3ab3-4932-b90a-e6588e2abab4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.046746 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befb474-a184-4af8-a051-6a0aa6b9e8e4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:08.048208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befb474-a184-4af8-a051-6a0aa6b9e8e4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8befb474-a184-4af8-a051-6a0aa6b9e8e4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:08.062955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:08.063890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:08.065490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 264/511] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:08.066299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befb474-a184-4af8-a051-6a0aa6b9e8e4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.149564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c60ce-3ab3-4932-b90a-e6588e2abab4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.164676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befb474-a184-4af8-a051-6a0aa6b9e8e4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.177416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:08.183169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1d22b-ab60-4ce4-9ddf-d77c22387667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.183826 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1d22b-ab60-4ce4-9ddf-d77c22387667 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.184112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1d22b-ab60-4ce4-9ddf-d77c22387667 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.184650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1d22b-ab60-4ce4-9ddf-d77c22387667 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:08.185163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1d22b-ab60-4ce4-9ddf-d77c22387667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:08.185465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1d22b-ab60-4ce4-9ddf-d77c22387667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:08.186867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1d22b-ab60-4ce4-9ddf-d77c22387667 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:08.199033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 248/512] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:08.199154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e258ea-05e8-4fe8-8e38-d570529c8cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.199816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e258ea-05e8-4fe8-8e38-d570529c8cbe None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.200097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e258ea-05e8-4fe8-8e38-d570529c8cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.200213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1d22b-ab60-4ce4-9ddf-d77c22387667 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.200370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e258ea-05e8-4fe8-8e38-d570529c8cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:08.211517 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e258ea-05e8-4fe8-8e38-d570529c8cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:08.212962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e258ea-05e8-4fe8-8e38-d570529c8cbe None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:08.221536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f4e258ea-05e8-4fe8-8e38-d570529c8cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:08.230643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:08.237276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:08.238472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:08.238472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:08.238472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:08.238472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:08.238472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 265/513] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:08.238472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e258ea-05e8-4fe8-8e38-d570529c8cbe None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.302100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1d22b-ab60-4ce4-9ddf-d77c22387667 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.335223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b215b0e-de75-4d18-a38f-da34d8664603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.335893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b215b0e-de75-4d18-a38f-da34d8664603 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.336169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b215b0e-de75-4d18-a38f-da34d8664603 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.336707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b215b0e-de75-4d18-a38f-da34d8664603 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:08.337169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b215b0e-de75-4d18-a38f-da34d8664603 None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:08.337466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b215b0e-de75-4d18-a38f-da34d8664603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:08.338869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b215b0e-de75-4d18-a38f-da34d8664603 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:08.348447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 249/514] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:08.349252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b215b0e-de75-4d18-a38f-da34d8664603 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.349573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e258ea-05e8-4fe8-8e38-d570529c8cbe None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.384756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ec658-387b-4bf7-9ad5-2693685ad298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.385536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ec658-387b-4bf7-9ad5-2693685ad298 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.385810 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ec658-387b-4bf7-9ad5-2693685ad298 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.386105 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ec658-387b-4bf7-9ad5-2693685ad298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:08.396589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ec658-387b-4bf7-9ad5-2693685ad298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:08.398042 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ec658-387b-4bf7-9ad5-2693685ad298 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:08.402832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-767ec658-387b-4bf7-9ad5-2693685ad298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:08.415374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:08.416260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:08.417093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:08.417093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:08.417093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:08.417093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:08.417093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 266/515] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:08.417585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ec658-387b-4bf7-9ad5-2693685ad298 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.454369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b215b0e-de75-4d18-a38f-da34d8664603 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.468809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:08.488930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3fcfc-952f-4603-b7b2-a6f55654a2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.489592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3fcfc-952f-4603-b7b2-a6f55654a2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.489872 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3fcfc-952f-4603-b7b2-a6f55654a2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.490177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3fcfc-952f-4603-b7b2-a6f55654a2f0 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:08.491002 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3fcfc-952f-4603-b7b2-a6f55654a2f0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:08.491308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3fcfc-952f-4603-b7b2-a6f55654a2f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:08.492750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3fcfc-952f-4603-b7b2-a6f55654a2f0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:08.509635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 250/516] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 13 08:48:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:08.510496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3fcfc-952f-4603-b7b2-a6f55654a2f0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.518079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ec658-387b-4bf7-9ad5-2693685ad298 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.552085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd9ab1-b376-42c3-a8c4-d1ee10515994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.552754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd9ab1-b376-42c3-a8c4-d1ee10515994 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.553037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd9ab1-b376-42c3-a8c4-d1ee10515994 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.553318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd9ab1-b376-42c3-a8c4-d1ee10515994 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:08.554021 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd9ab1-b376-42c3-a8c4-d1ee10515994 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:08.555503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd9ab1-b376-42c3-a8c4-d1ee10515994 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:08.608506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3fcfc-952f-4603-b7b2-a6f55654a2f0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.618910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:08.641726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdb0d7-763f-4461-bbf0-1525fd7a4eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.642410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdb0d7-763f-4461-bbf0-1525fd7a4eaf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.642747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdb0d7-763f-4461-bbf0-1525fd7a4eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.643687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdb0d7-763f-4461-bbf0-1525fd7a4eaf None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:08.644404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdb0d7-763f-4461-bbf0-1525fd7a4eaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:08.644712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdb0d7-763f-4461-bbf0-1525fd7a4eaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:08.645684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 267/517] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:48:08 2022] POST /identity/v3/users => generated 247 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:08.646148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdb0d7-763f-4461-bbf0-1525fd7a4eaf None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:08.649206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd9ab1-b376-42c3-a8c4-d1ee10515994 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.658266 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 251/518] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:08.659178 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdb0d7-763f-4461-bbf0-1525fd7a4eaf None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.774131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd9ab1-b376-42c3-a8c4-d1ee10515994 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.775516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdb0d7-763f-4461-bbf0-1525fd7a4eaf None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.784396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:08.813830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18382b-8f7c-4648-ac45-bf9a84bca1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.815324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18382b-8f7c-4648-ac45-bf9a84bca1fa None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.815603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18382b-8f7c-4648-ac45-bf9a84bca1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.815940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18382b-8f7c-4648-ac45-bf9a84bca1fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:08.821072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd8a71-9099-4eec-9aa4-5db5eb454f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.821736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd8a71-9099-4eec-9aa4-5db5eb454f50 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.822069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd8a71-9099-4eec-9aa4-5db5eb454f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.822481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd8a71-9099-4eec-9aa4-5db5eb454f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:08.823065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd8a71-9099-4eec-9aa4-5db5eb454f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:08.823373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd8a71-9099-4eec-9aa4-5db5eb454f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:08.824729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd8a71-9099-4eec-9aa4-5db5eb454f50 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:08.828488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18382b-8f7c-4648-ac45-bf9a84bca1fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:08.829950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18382b-8f7c-4648-ac45-bf9a84bca1fa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:08.837640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 252/519] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:08.838371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd8a71-9099-4eec-9aa4-5db5eb454f50 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-bd18382b-8f7c-4648-ac45-bf9a84bca1fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:08.848636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:08.849509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:08.850461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:08.850461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:08.850461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:08.850461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:08.850461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 268/520] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:08.851670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18382b-8f7c-4648-ac45-bf9a84bca1fa None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:08.878712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:08.955843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd8a71-9099-4eec-9aa4-5db5eb454f50 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.964592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18382b-8f7c-4648-ac45-bf9a84bca1fa None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:08.992394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3895164-d093-4cca-a95d-9ab67aa8e634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:08.993026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3895164-d093-4cca-a95d-9ab67aa8e634 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:08.993314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3895164-d093-4cca-a95d-9ab67aa8e634 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:08.993590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3895164-d093-4cca-a95d-9ab67aa8e634 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.000149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163f8d6-4d63-4192-82bc-d1b98f713d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:09.001109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163f8d6-4d63-4192-82bc-d1b98f713d35 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:09.001553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163f8d6-4d63-4192-82bc-d1b98f713d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:09.001983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163f8d6-4d63-4192-82bc-d1b98f713d35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.009295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163f8d6-4d63-4192-82bc-d1b98f713d35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.010781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163f8d6-4d63-4192-82bc-d1b98f713d35 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8163f8d6-4d63-4192-82bc-d1b98f713d35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:09.026269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:09.027298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:09.028912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:09.028912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:09.028912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:09.028912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:09.028912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:09.028912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:09.028912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:09.028912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:09.028912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 269/521] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:09.031146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163f8d6-4d63-4192-82bc-d1b98f713d35 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:09.055915 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:09.067974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3895164-d093-4cca-a95d-9ab67aa8e634 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.069434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3895164-d093-4cca-a95d-9ab67aa8e634 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.100804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:09.135890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163f8d6-4d63-4192-82bc-d1b98f713d35 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:09.175179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0079695-4d61-495c-9cf4-360dac38a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:09.175972 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0079695-4d61-495c-9cf4-360dac38a2ef None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:09.176251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0079695-4d61-495c-9cf4-360dac38a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:09.176525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0079695-4d61-495c-9cf4-360dac38a2ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.182145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 253/522] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:09.183198 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3895164-d093-4cca-a95d-9ab67aa8e634 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:09.183709 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0079695-4d61-495c-9cf4-360dac38a2ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.185148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0079695-4d61-495c-9cf4-360dac38a2ef None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d0079695-4d61-495c-9cf4-360dac38a2ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:09.199500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:09.200382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:09.201236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:09.201236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:09.201236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:09.201236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:09.201236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:09.201236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:09.201236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:09.201236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:09.201236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 270/523] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:09.201839 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0079695-4d61-495c-9cf4-360dac38a2ef None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:09.285141 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:09.297753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3895164-d093-4cca-a95d-9ab67aa8e634 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:09.304848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0079695-4d61-495c-9cf4-360dac38a2ef None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:09.327303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:09.333234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2ea4-9764-485c-b256-4750fbbabc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:09.333899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2ea4-9764-485c-b256-4750fbbabc28 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:09.334385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2ea4-9764-485c-b256-4750fbbabc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:09.334577 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2ea4-9764-485c-b256-4750fbbabc28 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.339246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2ea4-9764-485c-b256-4750fbbabc28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:09.339246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2ea4-9764-485c-b256-4750fbbabc28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.339246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2ea4-9764-485c-b256-4750fbbabc28 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.346661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd60eb-4834-47db-be30-858bd73b8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:09.347307 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd60eb-4834-47db-be30-858bd73b8adc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:09.347586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd60eb-4834-47db-be30-858bd73b8adc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:09.347862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd60eb-4834-47db-be30-858bd73b8adc None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.348661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd60eb-4834-47db-be30-858bd73b8adc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:09.348990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd60eb-4834-47db-be30-858bd73b8adc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.350452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd60eb-4834-47db-be30-858bd73b8adc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.358312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 254/524] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:09.358312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2ea4-9764-485c-b256-4750fbbabc28 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:09.372434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 271/525] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:09.373439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd60eb-4834-47db-be30-858bd73b8adc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:09.471889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2ea4-9764-485c-b256-4750fbbabc28 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:09.479607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd60eb-4834-47db-be30-858bd73b8adc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:09.513081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ec873-6490-4182-ab9b-0b8e0da6ed94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:09.513449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b99169-a3f0-41be-b26a-4a27abe42479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:09.513871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ec873-6490-4182-ab9b-0b8e0da6ed94 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:09.514124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b99169-a3f0-41be-b26a-4a27abe42479 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:09.514222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ec873-6490-4182-ab9b-0b8e0da6ed94 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:09.514407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b99169-a3f0-41be-b26a-4a27abe42479 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:09.514500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ec873-6490-4182-ab9b-0b8e0da6ed94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.514683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b99169-a3f0-41be-b26a-4a27abe42479 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.515463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b99169-a3f0-41be-b26a-4a27abe42479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:09.515773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b99169-a3f0-41be-b26a-4a27abe42479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.517245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b99169-a3f0-41be-b26a-4a27abe42479 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.526440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:09.526931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ec873-6490-4182-ab9b-0b8e0da6ed94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.529000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ec873-6490-4182-ab9b-0b8e0da6ed94 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.535022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 272/526] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:09.535022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b99169-a3f0-41be-b26a-4a27abe42479 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-112ec873-6490-4182-ab9b-0b8e0da6ed94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:09.555747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:09.556602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:09.557482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:09.557482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:09.557482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:09.557482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:09.558544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 255/527] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:09.558544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ec873-6490-4182-ab9b-0b8e0da6ed94 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:09.607058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:09.641845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b99169-a3f0-41be-b26a-4a27abe42479 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:09.672750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ec873-6490-4182-ab9b-0b8e0da6ed94 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:09.675579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6be834-ecf3-4d2f-a7ea-235d24e82922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:09.676374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6be834-ecf3-4d2f-a7ea-235d24e82922 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:09.676654 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6be834-ecf3-4d2f-a7ea-235d24e82922 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:09.676926 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6be834-ecf3-4d2f-a7ea-235d24e82922 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.684680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6be834-ecf3-4d2f-a7ea-235d24e82922 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.686139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6be834-ecf3-4d2f-a7ea-235d24e82922 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3b6be834-ecf3-4d2f-a7ea-235d24e82922 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:09.704363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:09.705370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:09.706405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 273/528] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:09.706982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6be834-ecf3-4d2f-a7ea-235d24e82922 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:09.712721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ee16b-4467-4c5c-861e-cad9be7e3f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:09.713378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ee16b-4467-4c5c-861e-cad9be7e3f6f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:09.713661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ee16b-4467-4c5c-861e-cad9be7e3f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:09.713932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ee16b-4467-4c5c-861e-cad9be7e3f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.714795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ee16b-4467-4c5c-861e-cad9be7e3f6f None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=cf67fa55f4d64b7ebeeb1490453661e0) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:09.715104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ee16b-4467-4c5c-861e-cad9be7e3f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.716559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ee16b-4467-4c5c-861e-cad9be7e3f6f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.723635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 256/529] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/role_assignments?user.id=cf67fa55f4d64b7ebeeb1490453661e0&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:09.724251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ee16b-4467-4c5c-861e-cad9be7e3f6f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:09.799506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:09.853928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6be834-ecf3-4d2f-a7ea-235d24e82922 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:09.869606 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ee16b-4467-4c5c-861e-cad9be7e3f6f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:09.879240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:09.894664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111600b6-c9ef-4c19-97e8-28aa7b7e1beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:09.895411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111600b6-c9ef-4c19-97e8-28aa7b7e1beb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:09.895753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111600b6-c9ef-4c19-97e8-28aa7b7e1beb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:09.896086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111600b6-c9ef-4c19-97e8-28aa7b7e1beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.897236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111600b6-c9ef-4c19-97e8-28aa7b7e1beb None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=8af26e036f4e466ebb923771c1f5f502) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:09.897692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111600b6-c9ef-4c19-97e8-28aa7b7e1beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.899820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111600b6-c9ef-4c19-97e8-28aa7b7e1beb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.905678 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 274/530] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/role_assignments?user.id=8af26e036f4e466ebb923771c1f5f502&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:09.909932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111600b6-c9ef-4c19-97e8-28aa7b7e1beb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:09.914852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e8d9c-6d91-4eb5-bdee-718a5d76870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:09.915546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e8d9c-6d91-4eb5-bdee-718a5d76870c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:09.915873 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e8d9c-6d91-4eb5-bdee-718a5d76870c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:09.916451 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e8d9c-6d91-4eb5-bdee-718a5d76870c None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:09.917391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e8d9c-6d91-4eb5-bdee-718a5d76870c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:09.917991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e8d9c-6d91-4eb5-bdee-718a5d76870c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:09.919439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e8d9c-6d91-4eb5-bdee-718a5d76870c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:09.933092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 257/531] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:09.940637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e8d9c-6d91-4eb5-bdee-718a5d76870c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:10.095349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e8d9c-6d91-4eb5-bdee-718a5d76870c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:10.138699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111600b6-c9ef-4c19-97e8-28aa7b7e1beb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:10.146371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf4c4f-8d3e-414a-9282-e69988cd7526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:10.147909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf4c4f-8d3e-414a-9282-e69988cd7526 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:10.148365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf4c4f-8d3e-414a-9282-e69988cd7526 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:10.148815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf4c4f-8d3e-414a-9282-e69988cd7526 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:10.159100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf4c4f-8d3e-414a-9282-e69988cd7526 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:10.160831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf4c4f-8d3e-414a-9282-e69988cd7526 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-fedf4c4f-8d3e-414a-9282-e69988cd7526 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:10.181724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:10.182648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:10.183504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:10.183504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:10.183504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:10.183504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:10.183504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:10.183504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:10.183504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:10.183504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:10.185775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 258/532] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:10.191418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf4c4f-8d3e-414a-9282-e69988cd7526 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:10.197139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:10.218858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e1518-5deb-4d0d-87cd-70304404795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:10.219800 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e1518-5deb-4d0d-87cd-70304404795c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:10.220410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e1518-5deb-4d0d-87cd-70304404795c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:10.222246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e1518-5deb-4d0d-87cd-70304404795c None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:10.223245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e1518-5deb-4d0d-87cd-70304404795c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:10.223909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e1518-5deb-4d0d-87cd-70304404795c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:10.226310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e1518-5deb-4d0d-87cd-70304404795c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:10.254094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:10.259461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 275/533] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Jul 13 08:48:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:10.266246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e1518-5deb-4d0d-87cd-70304404795c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:10.412022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf4c4f-8d3e-414a-9282-e69988cd7526 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:10.449922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e1518-5deb-4d0d-87cd-70304404795c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:10.463518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d70777-c8fe-4d75-8d5b-5b80532a2fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:10.464190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d70777-c8fe-4d75-8d5b-5b80532a2fbd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:10.464478 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d70777-c8fe-4d75-8d5b-5b80532a2fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:10.464760 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d70777-c8fe-4d75-8d5b-5b80532a2fbd None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:10.465558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d70777-c8fe-4d75-8d5b-5b80532a2fbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:10.465956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d70777-c8fe-4d75-8d5b-5b80532a2fbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:10.467460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d70777-c8fe-4d75-8d5b-5b80532a2fbd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:10.499026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 259/534] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:10.505024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d70777-c8fe-4d75-8d5b-5b80532a2fbd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:10.512116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85083524-9b18-4be9-a98a-8a6fb4519081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:10.512767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85083524-9b18-4be9-a98a-8a6fb4519081 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:10.513051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85083524-9b18-4be9-a98a-8a6fb4519081 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:10.513329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85083524-9b18-4be9-a98a-8a6fb4519081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:10.568300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:10.584354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85083524-9b18-4be9-a98a-8a6fb4519081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:10.585928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85083524-9b18-4be9-a98a-8a6fb4519081 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:10.650413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:10.657656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d70777-c8fe-4d75-8d5b-5b80532a2fbd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:10.726330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fad076-8633-4433-b0b2-eb52918ad383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:10.727683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fad076-8633-4433-b0b2-eb52918ad383 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:10.727970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fad076-8633-4433-b0b2-eb52918ad383 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:10.728246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fad076-8633-4433-b0b2-eb52918ad383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:10.729082 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fad076-8633-4433-b0b2-eb52918ad383 None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=b3f12750f91645748b37e2b159a9bd35) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:10.729397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fad076-8633-4433-b0b2-eb52918ad383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:10.730866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fad076-8633-4433-b0b2-eb52918ad383 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:10.752661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 260/535] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:10 2022] GET /identity/v3/role_assignments?user.id=b3f12750f91645748b37e2b159a9bd35&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:10.753187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 276/536] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:10.759183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fad076-8633-4433-b0b2-eb52918ad383 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:10.846309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8544a-2cf9-48a2-8a1c-1811f65b8874 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:10.846912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8544a-2cf9-48a2-8a1c-1811f65b8874 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:10.846912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8544a-2cf9-48a2-8a1c-1811f65b8874 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:10.848473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 277/537] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:10.854141 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d2afa-1333-4bdf-b2c9-07c7526b4ed2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:10.854429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d2afa-1333-4bdf-b2c9-07c7526b4ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:10.854705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d2afa-1333-4bdf-b2c9-07c7526b4ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:10.899371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-749d2afa-1333-4bdf-b2c9-07c7526b4ed2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:10.904116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-749d2afa-1333-4bdf-b2c9-07c7526b4ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:10.928153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fad076-8633-4433-b0b2-eb52918ad383 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:10.983854 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ec689-5285-4d23-92fa-e0bd7fb26884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:10.984581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ec689-5285-4d23-92fa-e0bd7fb26884 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:10.984863 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ec689-5285-4d23-92fa-e0bd7fb26884 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:10.985138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ec689-5285-4d23-92fa-e0bd7fb26884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:11.004358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ec689-5285-4d23-92fa-e0bd7fb26884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:11.005858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ec689-5285-4d23-92fa-e0bd7fb26884 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-559ec689-5285-4d23-92fa-e0bd7fb26884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:11.035810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:11.036702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:11.037548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:11.037548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:11.037548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:11.037548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:11.037548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 261/538] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:11.044225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 278/539] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:48:11.050258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ec689-5285-4d23-92fa-e0bd7fb26884 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:11.050734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:11.084543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d2afa-1333-4bdf-b2c9-07c7526b4ed2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:11.100325 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:11.217123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ec689-5285-4d23-92fa-e0bd7fb26884 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:11.251215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d2afa-1333-4bdf-b2c9-07c7526b4ed2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:11.260571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d7ebd-26c2-4eec-87c3-3e7f14b5702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:11.261253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d7ebd-26c2-4eec-87c3-3e7f14b5702d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:11.261535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d7ebd-26c2-4eec-87c3-3e7f14b5702d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:11.261809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d7ebd-26c2-4eec-87c3-3e7f14b5702d None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:11.262562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d7ebd-26c2-4eec-87c3-3e7f14b5702d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:11.262874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d7ebd-26c2-4eec-87c3-3e7f14b5702d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:11.264248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d7ebd-26c2-4eec-87c3-3e7f14b5702d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:11.292145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 262/540] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:11.299730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d7ebd-26c2-4eec-87c3-3e7f14b5702d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:11.301883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e597c2-c84c-4f41-8205-bee08cf7c805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:11.302712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e597c2-c84c-4f41-8205-bee08cf7c805 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:11.302991 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e597c2-c84c-4f41-8205-bee08cf7c805 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:11.303264 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e597c2-c84c-4f41-8205-bee08cf7c805 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:11.314201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e597c2-c84c-4f41-8205-bee08cf7c805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b0e597c2-c84c-4f41-8205-bee08cf7c805 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:11.325276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:11.326282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:11.327134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 279/541] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:11.339088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e597c2-c84c-4f41-8205-bee08cf7c805 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:11.389294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:11.446650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d7ebd-26c2-4eec-87c3-3e7f14b5702d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:11.486137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c1f94-f47d-4542-be39-22eab62a216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:11.488031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c1f94-f47d-4542-be39-22eab62a216c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:11.488730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c1f94-f47d-4542-be39-22eab62a216c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:11.489415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c1f94-f47d-4542-be39-22eab62a216c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:11.498217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c1f94-f47d-4542-be39-22eab62a216c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:11.499975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c1f94-f47d-4542-be39-22eab62a216c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:11.509604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-910c1f94-f47d-4542-be39-22eab62a216c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:11.528274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:11.529156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:11.529989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:11.529989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:11.529989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:11.529989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:11.529989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:11.529989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:11.529989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:11.529989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:11.531217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 263/542] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:11.539578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c1f94-f47d-4542-be39-22eab62a216c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:11.579808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e597c2-c84c-4f41-8205-bee08cf7c805 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:11.654749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7259e-9297-4c0c-ab7e-7d8328bcc5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:11.655434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7259e-9297-4c0c-ab7e-7d8328bcc5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:11.655725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7259e-9297-4c0c-ab7e-7d8328bcc5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:11.656004 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7259e-9297-4c0c-ab7e-7d8328bcc5d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:11.656763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7259e-9297-4c0c-ab7e-7d8328bcc5d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:11.657070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7259e-9297-4c0c-ab7e-7d8328bcc5d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:11.659043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7259e-9297-4c0c-ab7e-7d8328bcc5d0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:11.667940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 280/543] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:11.675409 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7259e-9297-4c0c-ab7e-7d8328bcc5d0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:11.700433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c1f94-f47d-4542-be39-22eab62a216c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:11.731494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:11.737600 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c756555-c8d5-43a7-840a-ec037883710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:11.738284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c756555-c8d5-43a7-840a-ec037883710d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:11.738567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c756555-c8d5-43a7-840a-ec037883710d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:11.738839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c756555-c8d5-43a7-840a-ec037883710d None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:11.739618 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c756555-c8d5-43a7-840a-ec037883710d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:11.739923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c756555-c8d5-43a7-840a-ec037883710d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:11.741351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c756555-c8d5-43a7-840a-ec037883710d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:11.767048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 264/544] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:11.776706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c756555-c8d5-43a7-840a-ec037883710d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:11.844070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7259e-9297-4c0c-ab7e-7d8328bcc5d0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:11.917995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64612bf-8fbd-4c52-925b-4cf0c68507b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:11.918809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64612bf-8fbd-4c52-925b-4cf0c68507b4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:11.919089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64612bf-8fbd-4c52-925b-4cf0c68507b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:11.919364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64612bf-8fbd-4c52-925b-4cf0c68507b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:11.928157 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:11.946086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64612bf-8fbd-4c52-925b-4cf0c68507b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:11.948661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64612bf-8fbd-4c52-925b-4cf0c68507b4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:11.973364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c756555-c8d5-43a7-840a-ec037883710d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f64612bf-8fbd-4c52-925b-4cf0c68507b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:11.977133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:11.977989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:11.978859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:11.978859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:11.978859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:11.978859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:11.986693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 281/545] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:11.992187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64612bf-8fbd-4c52-925b-4cf0c68507b4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:12.013513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c77d5-ba8f-46da-89a4-951eda7964c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:12.014198 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c77d5-ba8f-46da-89a4-951eda7964c7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.014471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c77d5-ba8f-46da-89a4-951eda7964c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.014751 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c77d5-ba8f-46da-89a4-951eda7964c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.015581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c77d5-ba8f-46da-89a4-951eda7964c7 None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=12824152139841138f457dfe5c9758e5) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:12.015891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c77d5-ba8f-46da-89a4-951eda7964c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:12.019348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c77d5-ba8f-46da-89a4-951eda7964c7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:12.031395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 265/546] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:11 2022] GET /identity/v3/role_assignments?user.id=12824152139841138f457dfe5c9758e5&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:12.036339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f556c4-db51-44f6-87e1-4b4f86bb6158 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.036615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f556c4-db51-44f6-87e1-4b4f86bb6158 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.036891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f556c4-db51-44f6-87e1-4b4f86bb6158 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.038539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 266/547] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:12.044397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9f5da-e344-420f-957c-bcd5ba120001 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.044890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9f5da-e344-420f-957c-bcd5ba120001 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.053788 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9f5da-e344-420f-957c-bcd5ba120001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.076987 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-14f9f5da-e344-420f-957c-bcd5ba120001 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:12.081055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-14f9f5da-e344-420f-957c-bcd5ba120001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:12.178156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64612bf-8fbd-4c52-925b-4cf0c68507b4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:12.217241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:12.228315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a631e2-0521-4c73-9a7e-942f06aa9a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:12.232023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a631e2-0521-4c73-9a7e-942f06aa9a0a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.232023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a631e2-0521-4c73-9a7e-942f06aa9a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.232023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a631e2-0521-4c73-9a7e-942f06aa9a0a None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.232023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a631e2-0521-4c73-9a7e-942f06aa9a0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:12.232023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a631e2-0521-4c73-9a7e-942f06aa9a0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:12.232359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a631e2-0521-4c73-9a7e-942f06aa9a0a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:12.254588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 282/548] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:12.259936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a631e2-0521-4c73-9a7e-942f06aa9a0a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:12.265521 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 267/549] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:12.298071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9f5da-e344-420f-957c-bcd5ba120001 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:12.318200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:12.407310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a631e2-0521-4c73-9a7e-942f06aa9a0a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:12.447577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f694749-e0e1-4173-80bc-9243eac48a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:12.448375 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694749-e0e1-4173-80bc-9243eac48a9f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.448656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694749-e0e1-4173-80bc-9243eac48a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.448929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694749-e0e1-4173-80bc-9243eac48a9f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.456847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f694749-e0e1-4173-80bc-9243eac48a9f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:12.458314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f694749-e0e1-4173-80bc-9243eac48a9f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6f694749-e0e1-4173-80bc-9243eac48a9f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:12.473960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:12.475108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:12.476208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 283/550] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:12.476781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b2696-2225-4082-ae57-66f1674ad79d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.476962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b2696-2225-4082-ae57-66f1674ad79d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.477247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b2696-2225-4082-ae57-66f1674ad79d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.478377 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9f5da-e344-420f-957c-bcd5ba120001 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:12.487314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 284/551] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:12.487314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b2696-2225-4082-ae57-66f1674ad79d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:12.521993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50440115-4cbb-4bb2-acc5-cae68bf2f82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:12.523151 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440115-4cbb-4bb2-acc5-cae68bf2f82f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.523577 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440115-4cbb-4bb2-acc5-cae68bf2f82f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.524020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440115-4cbb-4bb2-acc5-cae68bf2f82f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.536092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50440115-4cbb-4bb2-acc5-cae68bf2f82f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-50440115-4cbb-4bb2-acc5-cae68bf2f82f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:12.544934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:12.546311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:12.547312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:12.555021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 268/552] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:12.556627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b77708-13db-49aa-9401-d14252f709a7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.557108 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b77708-13db-49aa-9401-d14252f709a7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.557543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b77708-13db-49aa-9401-d14252f709a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.576477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d6b77708-13db-49aa-9401-d14252f709a7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:12.577958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d6b77708-13db-49aa-9401-d14252f709a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:12.593566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b2696-2225-4082-ae57-66f1674ad79d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:12.606047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:12.609284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:12.630635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f7757-ca61-40d1-8024-bd6e92f3cf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:12.631301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f7757-ca61-40d1-8024-bd6e92f3cf16 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.631602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f7757-ca61-40d1-8024-bd6e92f3cf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.638548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f7757-ca61-40d1-8024-bd6e92f3cf16 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.638548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f7757-ca61-40d1-8024-bd6e92f3cf16 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:12.638548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f7757-ca61-40d1-8024-bd6e92f3cf16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:12.638548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f7757-ca61-40d1-8024-bd6e92f3cf16 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:12.657040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 285/553] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 13 08:48:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:12.657531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 269/554] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:12.657935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f7757-ca61-40d1-8024-bd6e92f3cf16 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:12.658909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b77708-13db-49aa-9401-d14252f709a7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:12.764618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f7757-ca61-40d1-8024-bd6e92f3cf16 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:12.778683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b77708-13db-49aa-9401-d14252f709a7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:12.802177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587dbc6-b60d-4c87-9786-bc28c9355cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:12.802850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587dbc6-b60d-4c87-9786-bc28c9355cee None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.803133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587dbc6-b60d-4c87-9786-bc28c9355cee None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.803408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587dbc6-b60d-4c87-9786-bc28c9355cee None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.804305 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587dbc6-b60d-4c87-9786-bc28c9355cee None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:12.804614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587dbc6-b60d-4c87-9786-bc28c9355cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:12.806182 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587dbc6-b60d-4c87-9786-bc28c9355cee None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:12.820470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 286/555] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:12.820470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587dbc6-b60d-4c87-9786-bc28c9355cee None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:12.821376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091da40-d317-4abb-8b8e-a1d6d1ad3b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:12.822299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091da40-d317-4abb-8b8e-a1d6d1ad3b47 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.822761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091da40-d317-4abb-8b8e-a1d6d1ad3b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.823213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091da40-d317-4abb-8b8e-a1d6d1ad3b47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.854386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:12.860393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:12.898524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091da40-d317-4abb-8b8e-a1d6d1ad3b47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:12.900348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091da40-d317-4abb-8b8e-a1d6d1ad3b47 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:12.928526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587dbc6-b60d-4c87-9786-bc28c9355cee None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:12.965301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d93123-59f9-4f70-b6e7-cbe0e0f879df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:12.966300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d93123-59f9-4f70-b6e7-cbe0e0f879df None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:12.966585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d93123-59f9-4f70-b6e7-cbe0e0f879df None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:12.966859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d93123-59f9-4f70-b6e7-cbe0e0f879df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:12.977744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d93123-59f9-4f70-b6e7-cbe0e0f879df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a1d93123-59f9-4f70-b6e7-cbe0e0f879df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:12.985902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:12.986919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:12.987812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 287/556] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:12.988496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d93123-59f9-4f70-b6e7-cbe0e0f879df None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.010533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 270/557] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:13.012008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091da40-d317-4abb-8b8e-a1d6d1ad3b47 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.084280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:13.088082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:13.103387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d93123-59f9-4f70-b6e7-cbe0e0f879df None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.121661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091da40-d317-4abb-8b8e-a1d6d1ad3b47 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.138296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741dde59-89a3-4580-8526-9ac8ce9b123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:13.139088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dde59-89a3-4580-8526-9ac8ce9b123c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.139370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dde59-89a3-4580-8526-9ac8ce9b123c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.139645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dde59-89a3-4580-8526-9ac8ce9b123c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.151380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741dde59-89a3-4580-8526-9ac8ce9b123c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:13.153015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741dde59-89a3-4580-8526-9ac8ce9b123c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:13.158680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb92f9e-e197-4d2d-a397-78816fbe3b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:13.159346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb92f9e-e197-4d2d-a397-78816fbe3b0a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.159953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb92f9e-e197-4d2d-a397-78816fbe3b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.159953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb92f9e-e197-4d2d-a397-78816fbe3b0a None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.160669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb92f9e-e197-4d2d-a397-78816fbe3b0a None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:13.160986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb92f9e-e197-4d2d-a397-78816fbe3b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:13.162404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb92f9e-e197-4d2d-a397-78816fbe3b0a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:13.176415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 271/558] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-741dde59-89a3-4580-8526-9ac8ce9b123c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:13.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:13.177531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:13.178474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:13.178474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:13.178474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:13.178474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:13.178693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb92f9e-e197-4d2d-a397-78816fbe3b0a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.178769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 288/559] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:13.178769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd6ca-1def-43e9-84d3-e9d074fc5942 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.178769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd6ca-1def-43e9-84d3-e9d074fc5942 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.178769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd6ca-1def-43e9-84d3-e9d074fc5942 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.178769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 289/560] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:13.178769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fd6ca-1def-43e9-84d3-e9d074fc5942 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.285724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb92f9e-e197-4d2d-a397-78816fbe3b0a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.293016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fd6ca-1def-43e9-84d3-e9d074fc5942 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.313564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:13.322136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efc66b-2ee6-4e9e-a926-6adc05a9b64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:13.322924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efc66b-2ee6-4e9e-a926-6adc05a9b64f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.326875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efc66b-2ee6-4e9e-a926-6adc05a9b64f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.326875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efc66b-2ee6-4e9e-a926-6adc05a9b64f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.329141 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c35ed4-a655-4114-b88e-53bf8886c465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:13.329932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c35ed4-a655-4114-b88e-53bf8886c465 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.330833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c35ed4-a655-4114-b88e-53bf8886c465 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.331117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c35ed4-a655-4114-b88e-53bf8886c465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.335090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efc66b-2ee6-4e9e-a926-6adc05a9b64f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:13.336449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efc66b-2ee6-4e9e-a926-6adc05a9b64f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:13.341766 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:13.351719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c35ed4-a655-4114-b88e-53bf8886c465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:13.353179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c35ed4-a655-4114-b88e-53bf8886c465 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-68efc66b-2ee6-4e9e-a926-6adc05a9b64f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:13.355165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:13.359446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:13.360332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:13.360332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:13.360332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:13.360332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:13.360332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 272/561] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:13.360332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efc66b-2ee6-4e9e-a926-6adc05a9b64f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d3c35ed4-a655-4114-b88e-53bf8886c465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:13.373102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:13.373992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:13.375045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:13.375045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:13.375045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:13.375045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:13.375045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 290/562] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:13.379976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b703-fc9c-4db3-9880-58c8820ac86c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.380278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b703-fc9c-4db3-9880-58c8820ac86c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.380676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b703-fc9c-4db3-9880-58c8820ac86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.397934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1c71b703-fc9c-4db3-9880-58c8820ac86c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:13.399171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1c71b703-fc9c-4db3-9880-58c8820ac86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:13.467354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efc66b-2ee6-4e9e-a926-6adc05a9b64f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.478250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 291/563] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:13.478250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71b703-fc9c-4db3-9880-58c8820ac86c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.519265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9b2b2-1af6-438a-a62a-2ae34355a300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:13.519932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9b2b2-1af6-438a-a62a-2ae34355a300 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.520227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9b2b2-1af6-438a-a62a-2ae34355a300 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.520480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9b2b2-1af6-438a-a62a-2ae34355a300 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.521253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9b2b2-1af6-438a-a62a-2ae34355a300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:13.521571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9b2b2-1af6-438a-a62a-2ae34355a300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:13.522966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9b2b2-1af6-438a-a62a-2ae34355a300 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:13.539441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 273/564] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:13.540305 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9b2b2-1af6-438a-a62a-2ae34355a300 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.579477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71b703-fc9c-4db3-9880-58c8820ac86c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.585667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:13.600867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:13.615057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefbc62-57de-480a-b795-c7cdb337d2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:13.615763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefbc62-57de-480a-b795-c7cdb337d2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.616044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefbc62-57de-480a-b795-c7cdb337d2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.616319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefbc62-57de-480a-b795-c7cdb337d2a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.617040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aefbc62-57de-480a-b795-c7cdb337d2a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:13.617347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aefbc62-57de-480a-b795-c7cdb337d2a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:13.618759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aefbc62-57de-480a-b795-c7cdb337d2a9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:13.634966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 292/565] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:13.635726 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefbc62-57de-480a-b795-c7cdb337d2a9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.647913 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9b2b2-1af6-438a-a62a-2ae34355a300 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.685068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48a6f9-f447-46ec-b24f-1e45142aeb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:13.685726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48a6f9-f447-46ec-b24f-1e45142aeb7a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.686126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48a6f9-f447-46ec-b24f-1e45142aeb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.686316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48a6f9-f447-46ec-b24f-1e45142aeb7a None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.687088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48a6f9-f447-46ec-b24f-1e45142aeb7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:13.687513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48a6f9-f447-46ec-b24f-1e45142aeb7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:13.688807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48a6f9-f447-46ec-b24f-1e45142aeb7a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:13.701483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 274/566] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:13.702391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48a6f9-f447-46ec-b24f-1e45142aeb7a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.752916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefbc62-57de-480a-b795-c7cdb337d2a9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.791081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaa9e4-bb5f-41a0-92ea-cff904c2dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:13.791882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa9e4-bb5f-41a0-92ea-cff904c2dd82 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.792165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa9e4-bb5f-41a0-92ea-cff904c2dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.792441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa9e4-bb5f-41a0-92ea-cff904c2dd82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.803437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:13.806419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48a6f9-f447-46ec-b24f-1e45142aeb7a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.807950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcaa9e4-bb5f-41a0-92ea-cff904c2dd82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ddcaa9e4-bb5f-41a0-92ea-cff904c2dd82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:13.818873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:13.819801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 293/567] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:13.826085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaa9e4-bb5f-41a0-92ea-cff904c2dd82 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.842211 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cccb7e-9e5e-4bf5-8a0f-8d31310e95c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:13.842994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cccb7e-9e5e-4bf5-8a0f-8d31310e95c0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.843298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cccb7e-9e5e-4bf5-8a0f-8d31310e95c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.843571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cccb7e-9e5e-4bf5-8a0f-8d31310e95c0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.850821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cccb7e-9e5e-4bf5-8a0f-8d31310e95c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:13.852796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cccb7e-9e5e-4bf5-8a0f-8d31310e95c0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-57cccb7e-9e5e-4bf5-8a0f-8d31310e95c0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:13.867608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:13.868624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 275/568] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:13.874840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cccb7e-9e5e-4bf5-8a0f-8d31310e95c0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:13.878794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:13.942573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaa9e4-bb5f-41a0-92ea-cff904c2dd82 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.977923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cccb7e-9e5e-4bf5-8a0f-8d31310e95c0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:13.981963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8c614-336d-48e4-9c5a-2476b518cf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:13.982999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8c614-336d-48e4-9c5a-2476b518cf8a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:13.983425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8c614-336d-48e4-9c5a-2476b518cf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:13.983842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8c614-336d-48e4-9c5a-2476b518cf8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:13.992089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8c614-336d-48e4-9c5a-2476b518cf8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:13.993699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8c614-336d-48e4-9c5a-2476b518cf8a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6ae8c614-336d-48e4-9c5a-2476b518cf8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:14.008549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:14.009542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:14.010614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:14.010614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:14.010614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:14.010614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:14.010614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:14.010614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:14.010614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:14.010614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:14.012270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0cbe9-a53a-4eb4-8009-d5bdc5bb67e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.013118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 294/569] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:14.013222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0cbe9-a53a-4eb4-8009-d5bdc5bb67e5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.013430 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0cbe9-a53a-4eb4-8009-d5bdc5bb67e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.013755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0cbe9-a53a-4eb4-8009-d5bdc5bb67e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.014490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8c614-336d-48e4-9c5a-2476b518cf8a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.024898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0cbe9-a53a-4eb4-8009-d5bdc5bb67e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.026366 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0cbe9-a53a-4eb4-8009-d5bdc5bb67e5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-efa0cbe9-a53a-4eb4-8009-d5bdc5bb67e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:14.040973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:14.041952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 276/570] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 13 08:48:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:14.043017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0cbe9-a53a-4eb4-8009-d5bdc5bb67e5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.066474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:14.126271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:14.130718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8c614-336d-48e4-9c5a-2476b518cf8a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:14.147929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0cbe9-a53a-4eb4-8009-d5bdc5bb67e5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:14.171715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543706c-399a-4a78-b82f-cee6f56fecd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.172782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543706c-399a-4a78-b82f-cee6f56fecd0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.173515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543706c-399a-4a78-b82f-cee6f56fecd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.174980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543706c-399a-4a78-b82f-cee6f56fecd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.176054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543706c-399a-4a78-b82f-cee6f56fecd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:14.176862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543706c-399a-4a78-b82f-cee6f56fecd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.178878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543706c-399a-4a78-b82f-cee6f56fecd0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.184087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e79af0-c128-4cda-acf1-fd3c54ccd10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.184754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e79af0-c128-4cda-acf1-fd3c54ccd10e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.185036 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e79af0-c128-4cda-acf1-fd3c54ccd10e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.185314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e79af0-c128-4cda-acf1-fd3c54ccd10e None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.186174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e79af0-c128-4cda-acf1-fd3c54ccd10e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:14.186485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e79af0-c128-4cda-acf1-fd3c54ccd10e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.187905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e79af0-c128-4cda-acf1-fd3c54ccd10e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.188408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 295/571] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:14.189186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543706c-399a-4a78-b82f-cee6f56fecd0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.205685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 277/572] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:14.206617 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e79af0-c128-4cda-acf1-fd3c54ccd10e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.281841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:14.289464 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543706c-399a-4a78-b82f-cee6f56fecd0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:14.304098 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e79af0-c128-4cda-acf1-fd3c54ccd10e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:14.324633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70d4b9-546b-45cb-8f60-dc1d8064cc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.325287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70d4b9-546b-45cb-8f60-dc1d8064cc06 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.325567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70d4b9-546b-45cb-8f60-dc1d8064cc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.325841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70d4b9-546b-45cb-8f60-dc1d8064cc06 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.326657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70d4b9-546b-45cb-8f60-dc1d8064cc06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:14.326961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70d4b9-546b-45cb-8f60-dc1d8064cc06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.328403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70d4b9-546b-45cb-8f60-dc1d8064cc06 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.341696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 296/573] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:14.343011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69294579-bf7e-4f16-a4cf-b5f9d3eaccc3 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.343295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69294579-bf7e-4f16-a4cf-b5f9d3eaccc3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.343599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69294579-bf7e-4f16-a4cf-b5f9d3eaccc3 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.344798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 297/574] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:14.345479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69294579-bf7e-4f16-a4cf-b5f9d3eaccc3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.348165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffddd13-a5a2-4299-8b35-d32305aebce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.348847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffddd13-a5a2-4299-8b35-d32305aebce8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.349137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffddd13-a5a2-4299-8b35-d32305aebce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.349421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffddd13-a5a2-4299-8b35-d32305aebce8 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.350297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffddd13-a5a2-4299-8b35-d32305aebce8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:14.350612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffddd13-a5a2-4299-8b35-d32305aebce8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.352050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffddd13-a5a2-4299-8b35-d32305aebce8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.370761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 278/575] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:14.371828 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffddd13-a5a2-4299-8b35-d32305aebce8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.372404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:14.451207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69294579-bf7e-4f16-a4cf-b5f9d3eaccc3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:14.471866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffddd13-a5a2-4299-8b35-d32305aebce8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:14.484497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab310a36-74ca-438c-bebb-660505427456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.485282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310a36-74ca-438c-bebb-660505427456 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.485565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310a36-74ca-438c-bebb-660505427456 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.485838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310a36-74ca-438c-bebb-660505427456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.496463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab310a36-74ca-438c-bebb-660505427456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.497901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab310a36-74ca-438c-bebb-660505427456 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.505766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee567c-6f2a-448f-8a0b-107345402e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.506591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee567c-6f2a-448f-8a0b-107345402e06 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.506876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee567c-6f2a-448f-8a0b-107345402e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.507148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee567c-6f2a-448f-8a0b-107345402e06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.509925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ab310a36-74ca-438c-bebb-660505427456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:14.516958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:14.530070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:14.531160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:14.531160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:14.531160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:14.531160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:14.531436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee567c-6f2a-448f-8a0b-107345402e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.531524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 298/576] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:14.531524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab310a36-74ca-438c-bebb-660505427456 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.531668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee567c-6f2a-448f-8a0b-107345402e06 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f0ee567c-6f2a-448f-8a0b-107345402e06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:14.540526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:14.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:14.542594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:14.542594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:14.542594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:14.542594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:14.542594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 279/577] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:14.543081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185ae0c-b60d-4f95-9058-af3c194359b2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.543363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185ae0c-b60d-4f95-9058-af3c194359b2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.543636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185ae0c-b60d-4f95-9058-af3c194359b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.561423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1185ae0c-b60d-4f95-9058-af3c194359b2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:14.562662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1185ae0c-b60d-4f95-9058-af3c194359b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:14.598533 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:14.622608 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab310a36-74ca-438c-bebb-660505427456 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:14.629805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 280/578] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:14.630668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185ae0c-b60d-4f95-9058-af3c194359b2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.670249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1408c0d-423f-4b0a-ac0e-f698012026ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.671159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1408c0d-423f-4b0a-ac0e-f698012026ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.671447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1408c0d-423f-4b0a-ac0e-f698012026ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.671726 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1408c0d-423f-4b0a-ac0e-f698012026ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.672554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1408c0d-423f-4b0a-ac0e-f698012026ec None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=906fc971a5ee4ffd9503ab13d46cd605) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:14.672867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1408c0d-423f-4b0a-ac0e-f698012026ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.674939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1408c0d-423f-4b0a-ac0e-f698012026ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.680342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 299/579] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/role_assignments?user.id=906fc971a5ee4ffd9503ab13d46cd605&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:14.681250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1408c0d-423f-4b0a-ac0e-f698012026ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.735378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185ae0c-b60d-4f95-9058-af3c194359b2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:14.755380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:14.774080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6dfcc-4f7d-46b8-9df3-8ea5e8c1a73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.774896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6dfcc-4f7d-46b8-9df3-8ea5e8c1a73f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.775182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6dfcc-4f7d-46b8-9df3-8ea5e8c1a73f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.775530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6dfcc-4f7d-46b8-9df3-8ea5e8c1a73f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.787263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6dfcc-4f7d-46b8-9df3-8ea5e8c1a73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.788736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6dfcc-4f7d-46b8-9df3-8ea5e8c1a73f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.807340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1408c0d-423f-4b0a-ac0e-f698012026ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-9fe6dfcc-4f7d-46b8-9df3-8ea5e8c1a73f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:14.808427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:14.809323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:14.810277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:14.810277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:14.810277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:14.810277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:14.810277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 281/580] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:14.810564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6dfcc-4f7d-46b8-9df3-8ea5e8c1a73f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.856132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374cdcd-0cf0-49f9-a3a8-56fbc2c40f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.856800 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374cdcd-0cf0-49f9-a3a8-56fbc2c40f07 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.857083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374cdcd-0cf0-49f9-a3a8-56fbc2c40f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.857358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374cdcd-0cf0-49f9-a3a8-56fbc2c40f07 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.858119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374cdcd-0cf0-49f9-a3a8-56fbc2c40f07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:14.858431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374cdcd-0cf0-49f9-a3a8-56fbc2c40f07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.859812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374cdcd-0cf0-49f9-a3a8-56fbc2c40f07 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.867162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:14.872140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 300/581] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:14.872931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374cdcd-0cf0-49f9-a3a8-56fbc2c40f07 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.932508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6dfcc-4f7d-46b8-9df3-8ea5e8c1a73f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:14.966430 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142245a7-aa70-435f-a40f-b95ccb33123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:14.967094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142245a7-aa70-435f-a40f-b95ccb33123e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:14.967377 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142245a7-aa70-435f-a40f-b95ccb33123e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:14.967650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142245a7-aa70-435f-a40f-b95ccb33123e None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:14.968372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142245a7-aa70-435f-a40f-b95ccb33123e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:14.968678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142245a7-aa70-435f-a40f-b95ccb33123e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:14.970066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142245a7-aa70-435f-a40f-b95ccb33123e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:14.982816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 282/582] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:14.983471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142245a7-aa70-435f-a40f-b95ccb33123e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:14.989181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374cdcd-0cf0-49f9-a3a8-56fbc2c40f07 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.015485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:15.028168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdaab7-de73-4ea0-a081-a8b763c33e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.028964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdaab7-de73-4ea0-a081-a8b763c33e75 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.029246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdaab7-de73-4ea0-a081-a8b763c33e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.029519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdaab7-de73-4ea0-a081-a8b763c33e75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.040966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdaab7-de73-4ea0-a081-a8b763c33e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-74fdaab7-de73-4ea0-a081-a8b763c33e75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:15.049067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:15.050066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:15.050917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 301/583] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:15.051624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdaab7-de73-4ea0-a081-a8b763c33e75 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.093395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142245a7-aa70-435f-a40f-b95ccb33123e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.114100 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:15.128306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80902cdb-1309-46a5-b34b-557fef809052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.129288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80902cdb-1309-46a5-b34b-557fef809052 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.129556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80902cdb-1309-46a5-b34b-557fef809052 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.129824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80902cdb-1309-46a5-b34b-557fef809052 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.130571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80902cdb-1309-46a5-b34b-557fef809052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:15.130873 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80902cdb-1309-46a5-b34b-557fef809052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:15.132249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80902cdb-1309-46a5-b34b-557fef809052 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:15.144416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 283/584] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:15.145222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80902cdb-1309-46a5-b34b-557fef809052 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.162085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdaab7-de73-4ea0-a081-a8b763c33e75 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.196760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe9172-e9f3-454b-a21f-cd95d92b0a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.197565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe9172-e9f3-454b-a21f-cd95d92b0a06 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.197852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe9172-e9f3-454b-a21f-cd95d92b0a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.198162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe9172-e9f3-454b-a21f-cd95d92b0a06 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.206811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe9172-e9f3-454b-a21f-cd95d92b0a06 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:15.208275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe9172-e9f3-454b-a21f-cd95d92b0a06 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3bfe9172-e9f3-454b-a21f-cd95d92b0a06 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:15.222538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:15.223494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:15.224445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 302/585] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:15.225047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe9172-e9f3-454b-a21f-cd95d92b0a06 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.247381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:15.257893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80902cdb-1309-46a5-b34b-557fef809052 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.295322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1448e3-4972-4da9-94c2-cd5acca0cbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.296373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1448e3-4972-4da9-94c2-cd5acca0cbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.296654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1448e3-4972-4da9-94c2-cd5acca0cbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.296930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1448e3-4972-4da9-94c2-cd5acca0cbb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.305780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1448e3-4972-4da9-94c2-cd5acca0cbb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:15.307999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1448e3-4972-4da9-94c2-cd5acca0cbb0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-5b1448e3-4972-4da9-94c2-cd5acca0cbb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:15.326408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:15.327323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:15.328203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:15.328203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:15.328203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:15.328203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:15.328203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:15.328203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:15.328203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:15.328203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:15.328203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 284/586] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:15.328705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1448e3-4972-4da9-94c2-cd5acca0cbb0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.339131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe9172-e9f3-454b-a21f-cd95d92b0a06 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.356500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:15.373240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f660d7-54e2-4535-91b4-ed20df72050f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.374560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f660d7-54e2-4535-91b4-ed20df72050f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.374560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f660d7-54e2-4535-91b4-ed20df72050f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.374560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f660d7-54e2-4535-91b4-ed20df72050f None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.375262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f660d7-54e2-4535-91b4-ed20df72050f None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:15.375563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f660d7-54e2-4535-91b4-ed20df72050f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:15.377457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f660d7-54e2-4535-91b4-ed20df72050f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:15.386312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 303/587] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:15.387319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f660d7-54e2-4535-91b4-ed20df72050f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.446073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1448e3-4972-4da9-94c2-cd5acca0cbb0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.482325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b5234-a067-4d46-9dd5-cc25d9299253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.483112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b5234-a067-4d46-9dd5-cc25d9299253 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.483405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b5234-a067-4d46-9dd5-cc25d9299253 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.483669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b5234-a067-4d46-9dd5-cc25d9299253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.490487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b5234-a067-4d46-9dd5-cc25d9299253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:15.491569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f660d7-54e2-4535-91b4-ed20df72050f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.491911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b5234-a067-4d46-9dd5-cc25d9299253 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-202b5234-a067-4d46-9dd5-cc25d9299253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:15.506918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:15.507963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:15.509051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:15.509051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:15.509051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:15.509051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:15.509051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:15.509051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:15.509051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:15.509051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:15.512985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 285/588] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:15.513726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b5234-a067-4d46-9dd5-cc25d9299253 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.526100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:15.531458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9160fca-ff79-478d-b2fb-42a903d21ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.532128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9160fca-ff79-478d-b2fb-42a903d21ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.532414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9160fca-ff79-478d-b2fb-42a903d21ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.532690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9160fca-ff79-478d-b2fb-42a903d21ef4 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.533710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9160fca-ff79-478d-b2fb-42a903d21ef4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:15.534020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9160fca-ff79-478d-b2fb-42a903d21ef4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:15.535552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9160fca-ff79-478d-b2fb-42a903d21ef4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:15.558858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 304/589] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 13 08:48:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:15.558858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9160fca-ff79-478d-b2fb-42a903d21ef4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.596290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:15.622051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b5234-a067-4d46-9dd5-cc25d9299253 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.659213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a437e-697a-42f8-8734-fee02f36b26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.659846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9160fca-ff79-478d-b2fb-42a903d21ef4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.660298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a437e-697a-42f8-8734-fee02f36b26a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.660720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a437e-697a-42f8-8734-fee02f36b26a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.661134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a437e-697a-42f8-8734-fee02f36b26a None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.662067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a437e-697a-42f8-8734-fee02f36b26a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:15.662540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a437e-697a-42f8-8734-fee02f36b26a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:15.663961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a437e-697a-42f8-8734-fee02f36b26a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:15.681277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 286/590] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:15.681277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a437e-697a-42f8-8734-fee02f36b26a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.700759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222ea99-e924-4b6f-8216-e81a3d8030ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.701556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222ea99-e924-4b6f-8216-e81a3d8030ee None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.701837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222ea99-e924-4b6f-8216-e81a3d8030ee None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.702139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222ea99-e924-4b6f-8216-e81a3d8030ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.712435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222ea99-e924-4b6f-8216-e81a3d8030ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:15.713884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222ea99-e924-4b6f-8216-e81a3d8030ee None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0222ea99-e924-4b6f-8216-e81a3d8030ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:15.731955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:15.732853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:15.733750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:15.733750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:15.733750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:15.733750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:15.733750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 305/591] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:15.734019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222ea99-e924-4b6f-8216-e81a3d8030ee None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.755984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:15.796405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a437e-697a-42f8-8734-fee02f36b26a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.835053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b3f8b-106c-45bb-a0c1-f2c346dcced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.835716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b3f8b-106c-45bb-a0c1-f2c346dcced9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.836001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b3f8b-106c-45bb-a0c1-f2c346dcced9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.836282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b3f8b-106c-45bb-a0c1-f2c346dcced9 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.837211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:15.838108 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b3f8b-106c-45bb-a0c1-f2c346dcced9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:15.838508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b3f8b-106c-45bb-a0c1-f2c346dcced9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:15.839999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b3f8b-106c-45bb-a0c1-f2c346dcced9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:15.846721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222ea99-e924-4b6f-8216-e81a3d8030ee None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:15.857553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 287/592] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:15.857553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b3f8b-106c-45bb-a0c1-f2c346dcced9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.891452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba461647-3269-404b-b2bf-b54f171510b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:15.892249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba461647-3269-404b-b2bf-b54f171510b1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:15.893167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba461647-3269-404b-b2bf-b54f171510b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:15.893420 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba461647-3269-404b-b2bf-b54f171510b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:15.903824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba461647-3269-404b-b2bf-b54f171510b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:15.905282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba461647-3269-404b-b2bf-b54f171510b1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ba461647-3269-404b-b2bf-b54f171510b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:15.924143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:15.925060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:15.925961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:15.925961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:15.925961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:15.925961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:15.925961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 306/593] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:15.926704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba461647-3269-404b-b2bf-b54f171510b1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:15.975321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b3f8b-106c-45bb-a0c1-f2c346dcced9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:16.003942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:16.009785 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b9ca9-2b4a-4e1d-befc-b8ace4425c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:16.010733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b9ca9-2b4a-4e1d-befc-b8ace4425c67 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:16.011041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b9ca9-2b4a-4e1d-befc-b8ace4425c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:16.011317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b9ca9-2b4a-4e1d-befc-b8ace4425c67 None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/cf67fa55f4d64b7ebeeb1490453661e0/roles/3ec600afe034440d873fe4834644b63b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:16.026978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b9ca9-2b4a-4e1d-befc-b8ace4425c67 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=cf67fa55f4d64b7ebeeb1490453661e0, role_id=3ec600afe034440d873fe4834644b63b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:16.028540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b9ca9-2b4a-4e1d-befc-b8ace4425c67 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:16.036733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba461647-3269-404b-b2bf-b54f171510b1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:16.070941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d3644-25fc-4b5e-9830-0601f7f5f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:16.071553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d3644-25fc-4b5e-9830-0601f7f5f8ab None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:16.071839 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d3644-25fc-4b5e-9830-0601f7f5f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:16.072113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d3644-25fc-4b5e-9830-0601f7f5f8ab None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:16.072829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d3644-25fc-4b5e-9830-0601f7f5f8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:16.073139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d3644-25fc-4b5e-9830-0601f7f5f8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:16.074576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d3644-25fc-4b5e-9830-0601f7f5f8ab None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:16.086366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 307/594] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:16.087113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d3644-25fc-4b5e-9830-0601f7f5f8ab None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:16.089551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:16.099093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 288/595] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:48:15 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/cf67fa55f4d64b7ebeeb1490453661e0/roles/3ec600afe034440d873fe4834644b63b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:48:16.100206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b9ca9-2b4a-4e1d-befc-b8ace4425c67 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:16.272440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b9ca9-2b4a-4e1d-befc-b8ace4425c67 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:16.273670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d3644-25fc-4b5e-9830-0601f7f5f8ab None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:16.317912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af9df-ed80-4887-a77a-479fa14ea586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:16.318304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53653b39-ac34-476d-a8dd-3bb47296e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:16.319188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af9df-ed80-4887-a77a-479fa14ea586 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:16.319318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53653b39-ac34-476d-a8dd-3bb47296e72b None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:16.319483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53653b39-ac34-476d-a8dd-3bb47296e72b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:16.319730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53653b39-ac34-476d-a8dd-3bb47296e72b None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/8af26e036f4e466ebb923771c1f5f502/roles/3ec600afe034440d873fe4834644b63b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:16.320148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af9df-ed80-4887-a77a-479fa14ea586 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:16.320692 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af9df-ed80-4887-a77a-479fa14ea586 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:16.321862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af9df-ed80-4887-a77a-479fa14ea586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:16.324299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af9df-ed80-4887-a77a-479fa14ea586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:16.326929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af9df-ed80-4887-a77a-479fa14ea586 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:16.337879 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53653b39-ac34-476d-a8dd-3bb47296e72b None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=8af26e036f4e466ebb923771c1f5f502, role_id=3ec600afe034440d873fe4834644b63b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:16.339399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53653b39-ac34-476d-a8dd-3bb47296e72b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:16.352085 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 289/596] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:16.353146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af9df-ed80-4887-a77a-479fa14ea586 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:16.398275 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:16.417670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 308/597] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:48:16 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/8af26e036f4e466ebb923771c1f5f502/roles/3ec600afe034440d873fe4834644b63b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:48:16.419864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53653b39-ac34-476d-a8dd-3bb47296e72b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:16.516214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af9df-ed80-4887-a77a-479fa14ea586 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:16.541232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:16.576899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee741829-d63d-40a3-b616-3095534351c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:16.577823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee741829-d63d-40a3-b616-3095534351c1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:16.578160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee741829-d63d-40a3-b616-3095534351c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:16.578435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee741829-d63d-40a3-b616-3095534351c1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:16.588172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee741829-d63d-40a3-b616-3095534351c1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:16.589849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee741829-d63d-40a3-b616-3095534351c1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ee741829-d63d-40a3-b616-3095534351c1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:16.609186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:16.610245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:16.611742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:16.611742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:16.611742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:16.611742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:16.611742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:16.611742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:16.611742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:16.611742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:16.611742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 08:48:16.611742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:16.612836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 290/598] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:16.617008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee741829-d63d-40a3-b616-3095534351c1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:16.634443 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53653b39-ac34-476d-a8dd-3bb47296e72b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:16.675731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ff93-43e1-4298-aabf-9d53ca241b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:16.676894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ff93-43e1-4298-aabf-9d53ca241b96 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:16.677532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ff93-43e1-4298-aabf-9d53ca241b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:16.678011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ff93-43e1-4298-aabf-9d53ca241b96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:16.685707 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00ff93-43e1-4298-aabf-9d53ca241b96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:16.687483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00ff93-43e1-4298-aabf-9d53ca241b96 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2a00ff93-43e1-4298-aabf-9d53ca241b96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:16.704287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:16.705837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:16.706791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:16.706791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:16.706791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:16.706791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:16.706791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:16.706791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:16.706791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:16.706791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:16.713400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 309/599] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:16.715638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ff93-43e1-4298-aabf-9d53ca241b96 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:16.749538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:16.753750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee741829-d63d-40a3-b616-3095534351c1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:16.801156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77b711-f596-4995-a896-8d0dd4420144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:16.802634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77b711-f596-4995-a896-8d0dd4420144 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:16.803521 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77b711-f596-4995-a896-8d0dd4420144 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:16.804012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77b711-f596-4995-a896-8d0dd4420144 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:16.805267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77b711-f596-4995-a896-8d0dd4420144 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:16.805729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77b711-f596-4995-a896-8d0dd4420144 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:16.808471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77b711-f596-4995-a896-8d0dd4420144 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:16.826754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ff93-43e1-4298-aabf-9d53ca241b96 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:16.834270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 291/600] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Jul 13 08:48:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:16.836435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b0b42-b29d-4407-a8f4-a540df330644 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:16.837322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b0b42-b29d-4407-a8f4-a540df330644 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:16.837759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b0b42-b29d-4407-a8f4-a540df330644 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:16.840302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 292/601] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:16.841186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b0b42-b29d-4407-a8f4-a540df330644 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:16.845980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:16.869692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952c17e-caee-4421-b209-9b86e251f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:16.870575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952c17e-caee-4421-b209-9b86e251f97a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:16.870948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952c17e-caee-4421-b209-9b86e251f97a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:16.871318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952c17e-caee-4421-b209-9b86e251f97a None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:16.872202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952c17e-caee-4421-b209-9b86e251f97a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:16.872604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952c17e-caee-4421-b209-9b86e251f97a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:16.874434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952c17e-caee-4421-b209-9b86e251f97a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:16.895249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 310/602] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:16.895249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fc0b6-ff51-4ab5-8e6d-3c4b2bc66b93 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:16.895249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fc0b6-ff51-4ab5-8e6d-3c4b2bc66b93 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:16.895249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fc0b6-ff51-4ab5-8e6d-3c4b2bc66b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:16.913241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-124fc0b6-ff51-4ab5-8e6d-3c4b2bc66b93 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:16.914602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-124fc0b6-ff51-4ab5-8e6d-3c4b2bc66b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:16.979799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b0b42-b29d-4407-a8f4-a540df330644 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:16.998150 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 311/603] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:16.999202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fc0b6-ff51-4ab5-8e6d-3c4b2bc66b93 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:17.021371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3766-716f-4766-925f-9b5ce929799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:17.022738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3766-716f-4766-925f-9b5ce929799e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:17.023107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3766-716f-4766-925f-9b5ce929799e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:17.023453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3766-716f-4766-925f-9b5ce929799e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:17.034431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b3766-716f-4766-925f-9b5ce929799e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:17.035902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b3766-716f-4766-925f-9b5ce929799e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d35b3766-716f-4766-925f-9b5ce929799e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:17.054708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:17.055895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:17.057254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:17.057254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:17.057254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:17.057254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:17.057254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 293/604] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:17.057254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3766-716f-4766-925f-9b5ce929799e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:17.070683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:17.141882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fc0b6-ff51-4ab5-8e6d-3c4b2bc66b93 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:17.145461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:17.166717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3766-716f-4766-925f-9b5ce929799e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:17.183432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca2c8f-2485-4f61-8945-5c0513858276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:17.184645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca2c8f-2485-4f61-8945-5c0513858276 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:17.185129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca2c8f-2485-4f61-8945-5c0513858276 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:17.185632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca2c8f-2485-4f61-8945-5c0513858276 None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/b3f12750f91645748b37e2b159a9bd35/roles/3ec600afe034440d873fe4834644b63b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:17.201035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca2c8f-2485-4f61-8945-5c0513858276 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=b3f12750f91645748b37e2b159a9bd35, role_id=3ec600afe034440d873fe4834644b63b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:17.202715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21ed70-b8ac-42f2-839c-b8983a458029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:17.202856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca2c8f-2485-4f61-8945-5c0513858276 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:17.203681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21ed70-b8ac-42f2-839c-b8983a458029 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:17.204056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21ed70-b8ac-42f2-839c-b8983a458029 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:17.204424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21ed70-b8ac-42f2-839c-b8983a458029 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:17.215855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21ed70-b8ac-42f2-839c-b8983a458029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ca21ed70-b8ac-42f2-839c-b8983a458029 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:17.223910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:17.231264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 294/605] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:17.232287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21ed70-b8ac-42f2-839c-b8983a458029 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:17.298444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 312/606] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:48:16 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/b3f12750f91645748b37e2b159a9bd35/roles/3ec600afe034440d873fe4834644b63b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:48:17.301077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca2c8f-2485-4f61-8945-5c0513858276 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:17.343743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:17.410557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21ed70-b8ac-42f2-839c-b8983a458029 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:17.473644 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c85021-994f-4c77-bf34-e5212f5fbfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:17.474355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85021-994f-4c77-bf34-e5212f5fbfee None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:17.474638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85021-994f-4c77-bf34-e5212f5fbfee None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:17.474997 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85021-994f-4c77-bf34-e5212f5fbfee None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:17.475660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c85021-994f-4c77-bf34-e5212f5fbfee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:17.475966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c85021-994f-4c77-bf34-e5212f5fbfee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:17.477336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c85021-994f-4c77-bf34-e5212f5fbfee None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:17.490078 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 295/607] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:17.492343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca2c8f-2485-4f61-8945-5c0513858276 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:17.496340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c85021-994f-4c77-bf34-e5212f5fbfee None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:17.536330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931afec-4aa6-4577-86f0-7beda74e4be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:17.537244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931afec-4aa6-4577-86f0-7beda74e4be7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:17.537537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931afec-4aa6-4577-86f0-7beda74e4be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:17.537815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931afec-4aa6-4577-86f0-7beda74e4be7 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:17.539538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931afec-4aa6-4577-86f0-7beda74e4be7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:17.539856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931afec-4aa6-4577-86f0-7beda74e4be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:17.541245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931afec-4aa6-4577-86f0-7beda74e4be7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:17.553114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 313/608] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:17.558723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931afec-4aa6-4577-86f0-7beda74e4be7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:17.581228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:17.613378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c85021-994f-4c77-bf34-e5212f5fbfee None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:17.649918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:17.656986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07cd1b-7bf3-4156-91ff-ac9ab8f45b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:17.660221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07cd1b-7bf3-4156-91ff-ac9ab8f45b29 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:17.660221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07cd1b-7bf3-4156-91ff-ac9ab8f45b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:17.660221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07cd1b-7bf3-4156-91ff-ac9ab8f45b29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:17.669371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07cd1b-7bf3-4156-91ff-ac9ab8f45b29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:17.671740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07cd1b-7bf3-4156-91ff-ac9ab8f45b29 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-fa07cd1b-7bf3-4156-91ff-ac9ab8f45b29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:17.694065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:17.695113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:17.695992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:17.695992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:17.695992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:17.695992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:17.695992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:17.695992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:17.695992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:17.695992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:17.695992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 296/609] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:17.696465 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931afec-4aa6-4577-86f0-7beda74e4be7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:17.697769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07cd1b-7bf3-4156-91ff-ac9ab8f45b29 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:17.779438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f03fa8-3f2d-47fb-852a-4aa0be9a45c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:17.780305 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f03fa8-3f2d-47fb-852a-4aa0be9a45c0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:17.780582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f03fa8-3f2d-47fb-852a-4aa0be9a45c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:17.780903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f03fa8-3f2d-47fb-852a-4aa0be9a45c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:17.802702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f03fa8-3f2d-47fb-852a-4aa0be9a45c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:17.804171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f03fa8-3f2d-47fb-852a-4aa0be9a45c0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-01f03fa8-3f2d-47fb-852a-4aa0be9a45c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:17.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:17.824884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:17.825756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:17.825756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:17.825756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:17.825756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:17.825756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 314/610] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:17.831642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f03fa8-3f2d-47fb-852a-4aa0be9a45c0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:17.906576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:17.947287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f03fa8-3f2d-47fb-852a-4aa0be9a45c0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:17.952799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07cd1b-7bf3-4156-91ff-ac9ab8f45b29 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:17.989018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913f0b1-ffb7-40b5-84b4-6a9522beaefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:17.989738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913f0b1-ffb7-40b5-84b4-6a9522beaefc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:17.990021 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913f0b1-ffb7-40b5-84b4-6a9522beaefc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:17.990333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913f0b1-ffb7-40b5-84b4-6a9522beaefc None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:17.991513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913f0b1-ffb7-40b5-84b4-6a9522beaefc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:17.991815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913f0b1-ffb7-40b5-84b4-6a9522beaefc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:17.993194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913f0b1-ffb7-40b5-84b4-6a9522beaefc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:18.007636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 315/611] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:18.012787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913f0b1-ffb7-40b5-84b4-6a9522beaefc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:18.030226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacf497-0d0f-47d0-86f5-8df70054cdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:18.030894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf497-0d0f-47d0-86f5-8df70054cdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.031175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf497-0d0f-47d0-86f5-8df70054cdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.031450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf497-0d0f-47d0-86f5-8df70054cdf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.032243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacf497-0d0f-47d0-86f5-8df70054cdf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:18.032549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacf497-0d0f-47d0-86f5-8df70054cdf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:18.035099 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacf497-0d0f-47d0-86f5-8df70054cdf8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:18.058645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 297/612] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:18.064621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacf497-0d0f-47d0-86f5-8df70054cdf8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:18.138410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:18.218831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacf497-0d0f-47d0-86f5-8df70054cdf8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:18.231865 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913f0b1-ffb7-40b5-84b4-6a9522beaefc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:18.278984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:18.288653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880b0cb-ecd4-4761-abd6-a6a4d93cb7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:18.289717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880b0cb-ecd4-4761-abd6-a6a4d93cb7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.290200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880b0cb-ecd4-4761-abd6-a6a4d93cb7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.290632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880b0cb-ecd4-4761-abd6-a6a4d93cb7c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.294494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89054f17-6ca3-427c-9d0e-2358f7b07e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:18.294494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89054f17-6ca3-427c-9d0e-2358f7b07e20 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.294494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89054f17-6ca3-427c-9d0e-2358f7b07e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.294494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89054f17-6ca3-427c-9d0e-2358f7b07e20 None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/12824152139841138f457dfe5c9758e5/roles/3ec600afe034440d873fe4834644b63b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.299444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880b0cb-ecd4-4761-abd6-a6a4d93cb7c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:18.301126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880b0cb-ecd4-4761-abd6-a6a4d93cb7c8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4880b0cb-ecd4-4761-abd6-a6a4d93cb7c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:18.317680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:18.318705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:18.319633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:18.319633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:18.319633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:18.319633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:18.319633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:18.319633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:18.319633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:18.319633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:18.319633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:18.319633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:18.322648 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 316/613] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:18.327527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89054f17-6ca3-427c-9d0e-2358f7b07e20 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=12824152139841138f457dfe5c9758e5, role_id=3ec600afe034440d873fe4834644b63b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:18.328525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880b0cb-ecd4-4761-abd6-a6a4d93cb7c8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:18.329091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89054f17-6ca3-427c-9d0e-2358f7b07e20 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:18.422129 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 298/614] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:48:18 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/12824152139841138f457dfe5c9758e5/roles/3ec600afe034440d873fe4834644b63b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:48:18.441085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880b0cb-ecd4-4761-abd6-a6a4d93cb7c8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:18.480806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627be5c-d306-4332-8041-978a10d85bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:18.481831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627be5c-d306-4332-8041-978a10d85bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.482341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627be5c-d306-4332-8041-978a10d85bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.482805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627be5c-d306-4332-8041-978a10d85bb2 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.483866 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627be5c-d306-4332-8041-978a10d85bb2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:18.484373 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627be5c-d306-4332-8041-978a10d85bb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:18.486094 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627be5c-d306-4332-8041-978a10d85bb2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:18.506612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 317/615] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 13 08:48:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:18.514106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89054f17-6ca3-427c-9d0e-2358f7b07e20 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:18.546826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe0298-23ef-49dc-b3f2-5b3a6439e707 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.547397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe0298-23ef-49dc-b3f2-5b3a6439e707 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.547851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe0298-23ef-49dc-b3f2-5b3a6439e707 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.550278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 318/616] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:18.555558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adda97-83b9-4ae1-8d84-5f0a52b5b99d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.556098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adda97-83b9-4ae1-8d84-5f0a52b5b99d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.556578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adda97-83b9-4ae1-8d84-5f0a52b5b99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.583376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-98adda97-83b9-4ae1-8d84-5f0a52b5b99d None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:18.584960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-98adda97-83b9-4ae1-8d84-5f0a52b5b99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:18.632871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:18.665906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 319/617] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:48:18.697533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89054f17-6ca3-427c-9d0e-2358f7b07e20 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:18.698483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adda97-83b9-4ae1-8d84-5f0a52b5b99d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:18.699547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:18.742106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2cc3a0-8196-410d-8a37-38118776a6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:18.742979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2cc3a0-8196-410d-8a37-38118776a6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.743361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2cc3a0-8196-410d-8a37-38118776a6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.743732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2cc3a0-8196-410d-8a37-38118776a6d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.763456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2cc3a0-8196-410d-8a37-38118776a6d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:18.766068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2cc3a0-8196-410d-8a37-38118776a6d5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ca2cc3a0-8196-410d-8a37-38118776a6d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:18.791776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:18.792724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:18.793736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:18.793736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:18.793736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:18.793736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:18.798111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 299/618] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:18.800518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305afc-dff5-4c83-8514-5821128bddf6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.801012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305afc-dff5-4c83-8514-5821128bddf6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.801984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305afc-dff5-4c83-8514-5821128bddf6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.804030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 300/619] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:18.805299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305afc-dff5-4c83-8514-5821128bddf6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:18.821775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adda97-83b9-4ae1-8d84-5f0a52b5b99d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:18.860988 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187b6b9-8b9a-4b20-924f-66c6ed316d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:18.861941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187b6b9-8b9a-4b20-924f-66c6ed316d9f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.862311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187b6b9-8b9a-4b20-924f-66c6ed316d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.862677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187b6b9-8b9a-4b20-924f-66c6ed316d9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.876399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187b6b9-8b9a-4b20-924f-66c6ed316d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1187b6b9-8b9a-4b20-924f-66c6ed316d9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:18.886384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:18.887530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:18.888577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:18.890268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 320/620] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:18.891994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2657-2ff9-499e-adfe-f1ebd56a64f6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.893130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2657-2ff9-499e-adfe-f1ebd56a64f6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.893496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2657-2ff9-499e-adfe-f1ebd56a64f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.915431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7acc2657-2ff9-499e-adfe-f1ebd56a64f6 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:18.917121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7acc2657-2ff9-499e-adfe-f1ebd56a64f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:18.947282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305afc-dff5-4c83-8514-5821128bddf6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:18.951275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:18.994690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182486b-c0bf-4224-98ff-e17df186ff8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:18.995873 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182486b-c0bf-4224-98ff-e17df186ff8f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:18.996737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182486b-c0bf-4224-98ff-e17df186ff8f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:18.997486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182486b-c0bf-4224-98ff-e17df186ff8f None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:18.998403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182486b-c0bf-4224-98ff-e17df186ff8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:18.998905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182486b-c0bf-4224-98ff-e17df186ff8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:19.000436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182486b-c0bf-4224-98ff-e17df186ff8f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:19.001343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 321/621] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:19.003071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc2657-2ff9-499e-adfe-f1ebd56a64f6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:19.024087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 301/622] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:19.024985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182486b-c0bf-4224-98ff-e17df186ff8f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:19.044298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:19.147496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc2657-2ff9-499e-adfe-f1ebd56a64f6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:19.159511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182486b-c0bf-4224-98ff-e17df186ff8f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:19.193871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e2e17-3c2b-4c7a-a51c-b66085ed8d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:19.194691 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e2e17-3c2b-4c7a-a51c-b66085ed8d3b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.195044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e2e17-3c2b-4c7a-a51c-b66085ed8d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.195647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e2e17-3c2b-4c7a-a51c-b66085ed8d3b None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.196884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e2e17-3c2b-4c7a-a51c-b66085ed8d3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:19.197265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e2e17-3c2b-4c7a-a51c-b66085ed8d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:19.199308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e2e17-3c2b-4c7a-a51c-b66085ed8d3b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:19.206819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577ef79-03e6-4a86-af1d-0e01d6195975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:19.208139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577ef79-03e6-4a86-af1d-0e01d6195975 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.208596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577ef79-03e6-4a86-af1d-0e01d6195975 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.209051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577ef79-03e6-4a86-af1d-0e01d6195975 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.210111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 322/623] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:19.211156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e2e17-3c2b-4c7a-a51c-b66085ed8d3b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:19.223575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577ef79-03e6-4a86-af1d-0e01d6195975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-a577ef79-03e6-4a86-af1d-0e01d6195975 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:19.234596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:19.235494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:19.237181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:19.240448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 302/624] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:19.241645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577ef79-03e6-4a86-af1d-0e01d6195975 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:19.244595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:19.320439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:19.333480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e2e17-3c2b-4c7a-a51c-b66085ed8d3b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:19.353393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577ef79-03e6-4a86-af1d-0e01d6195975 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:19.381077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd81566-d022-4761-bb58-565ac9bc2d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:19.382175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd81566-d022-4761-bb58-565ac9bc2d59 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.382653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd81566-d022-4761-bb58-565ac9bc2d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.383161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd81566-d022-4761-bb58-565ac9bc2d59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.388867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7333-d9b1-432e-b15a-9f817104117b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:19.389770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7333-d9b1-432e-b15a-9f817104117b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.390213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7333-d9b1-432e-b15a-9f817104117b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.390559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7333-d9b1-432e-b15a-9f817104117b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.391162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd81566-d022-4761-bb58-565ac9bc2d59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:19.393520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd81566-d022-4761-bb58-565ac9bc2d59 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:19.402500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b7333-d9b1-432e-b15a-9f817104117b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:19.404072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b7333-d9b1-432e-b15a-9f817104117b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3fd81566-d022-4761-bb58-565ac9bc2d59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:19.410104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:19.410992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:19.411877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:19.411877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:19.411877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:19.411877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:19.411877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:19.411877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:19.411877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:19.411877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:19.414441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 323/625] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:19.415441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd81566-d022-4761-bb58-565ac9bc2d59 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-1b5b7333-d9b1-432e-b15a-9f817104117b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:19.428585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:19.429502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:19.430417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:19.430417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:19.430417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:19.430417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:19.430417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 303/626] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:19.431269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7333-d9b1-432e-b15a-9f817104117b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:19.512418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:19.529323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd81566-d022-4761-bb58-565ac9bc2d59 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:19.536791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7333-d9b1-432e-b15a-9f817104117b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:19.561580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:19.571657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c130be6-a17a-43b7-a5e6-2d9f3c2b06ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:19.572041 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c130be6-a17a-43b7-a5e6-2d9f3c2b06ae None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.572470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c130be6-a17a-43b7-a5e6-2d9f3c2b06ae None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.572937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c130be6-a17a-43b7-a5e6-2d9f3c2b06ae None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.573837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c130be6-a17a-43b7-a5e6-2d9f3c2b06ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:19.573952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1193d7-0b7a-49d6-88ef-8cccf00bff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:19.574416 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c130be6-a17a-43b7-a5e6-2d9f3c2b06ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:19.574789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1193d7-0b7a-49d6-88ef-8cccf00bff18 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.575165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1193d7-0b7a-49d6-88ef-8cccf00bff18 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.575541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1193d7-0b7a-49d6-88ef-8cccf00bff18 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.576405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c130be6-a17a-43b7-a5e6-2d9f3c2b06ae None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:19.576536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1193d7-0b7a-49d6-88ef-8cccf00bff18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:19.576909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1193d7-0b7a-49d6-88ef-8cccf00bff18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:19.578644 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1193d7-0b7a-49d6-88ef-8cccf00bff18 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:19.587386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 324/627] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:19.588827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c130be6-a17a-43b7-a5e6-2d9f3c2b06ae None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:19.596814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 304/628] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:19.597811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1193d7-0b7a-49d6-88ef-8cccf00bff18 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:19.720102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c130be6-a17a-43b7-a5e6-2d9f3c2b06ae None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:19.724488 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1193d7-0b7a-49d6-88ef-8cccf00bff18 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:19.758423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ccaa7-f97a-4f7f-8eca-0454a0c17cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:19.759251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ccaa7-f97a-4f7f-8eca-0454a0c17cae None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.759634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ccaa7-f97a-4f7f-8eca-0454a0c17cae None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.760012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ccaa7-f97a-4f7f-8eca-0454a0c17cae None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.760838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ccaa7-f97a-4f7f-8eca-0454a0c17cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:19.761246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ccaa7-f97a-4f7f-8eca-0454a0c17cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:19.762750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ccaa7-f97a-4f7f-8eca-0454a0c17cae None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:19.768491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da9c09-dcce-48f2-8e2c-e94f55a7a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:19.769281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da9c09-dcce-48f2-8e2c-e94f55a7a53f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.769557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da9c09-dcce-48f2-8e2c-e94f55a7a53f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.769824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da9c09-dcce-48f2-8e2c-e94f55a7a53f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.777236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 325/629] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:19.778320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ccaa7-f97a-4f7f-8eca-0454a0c17cae None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:19.782775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da9c09-dcce-48f2-8e2c-e94f55a7a53f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:19.788036 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da9c09-dcce-48f2-8e2c-e94f55a7a53f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:19.789857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-60da9c09-dcce-48f2-8e2c-e94f55a7a53f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:19.810898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:19.815545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:19.816471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:19.816471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:19.816471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:19.816471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:19.816471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 305/630] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:19.816471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da9c09-dcce-48f2-8e2c-e94f55a7a53f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:19.849807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:19.887949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ccaa7-f97a-4f7f-8eca-0454a0c17cae None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:19.912241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da9c09-dcce-48f2-8e2c-e94f55a7a53f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:19.925287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c2ebc-cf12-4037-9866-87a148c344b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:19.926313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c2ebc-cf12-4037-9866-87a148c344b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.926598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c2ebc-cf12-4037-9866-87a148c344b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.926889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c2ebc-cf12-4037-9866-87a148c344b7 None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/906fc971a5ee4ffd9503ab13d46cd605/roles/3ec600afe034440d873fe4834644b63b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.942777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c2ebc-cf12-4037-9866-87a148c344b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=906fc971a5ee4ffd9503ab13d46cd605, role_id=3ec600afe034440d873fe4834644b63b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:19.944230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c2ebc-cf12-4037-9866-87a148c344b7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:19.945541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d79b1-9b2b-40d0-bdf2-2deb813ae6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:19.946370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d79b1-9b2b-40d0-bdf2-2deb813ae6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.946644 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d79b1-9b2b-40d0-bdf2-2deb813ae6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.946921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d79b1-9b2b-40d0-bdf2-2deb813ae6f2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.953815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d79b1-9b2b-40d0-bdf2-2deb813ae6f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:19.955930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d79b1-9b2b-40d0-bdf2-2deb813ae6f2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-815d79b1-9b2b-40d0-bdf2-2deb813ae6f2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:19.970106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:19.971034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:19.971986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 306/631] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:19.972583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9235a-3ddc-4016-938b-505807ca7a42 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:19.973140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9235a-3ddc-4016-938b-505807ca7a42 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:19.973140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9235a-3ddc-4016-938b-505807ca7a42 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:19.976289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 307/632] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:19.976289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9235a-3ddc-4016-938b-505807ca7a42 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:20.019489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 326/633] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:48:19 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/906fc971a5ee4ffd9503ab13d46cd605/roles/3ec600afe034440d873fe4834644b63b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:48:20.019489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c2ebc-cf12-4037-9866-87a148c344b7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:20.087298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:20.090999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:20.098094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9235a-3ddc-4016-938b-505807ca7a42 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:20.143152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c2ebc-cf12-4037-9866-87a148c344b7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:20.151251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e3388-6c4d-4472-9f1a-6b1378b2f00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:20.151940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e3388-6c4d-4472-9f1a-6b1378b2f00f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.152219 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e3388-6c4d-4472-9f1a-6b1378b2f00f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.152490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e3388-6c4d-4472-9f1a-6b1378b2f00f None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.153855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e3388-6c4d-4472-9f1a-6b1378b2f00f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:20.154197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e3388-6c4d-4472-9f1a-6b1378b2f00f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:20.155639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e3388-6c4d-4472-9f1a-6b1378b2f00f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:20.176725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 308/634] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:20.176725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e8b65-b3c4-4bb1-aa05-b64c70bdc8bc None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.176725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e8b65-b3c4-4bb1-aa05-b64c70bdc8bc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.176725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e8b65-b3c4-4bb1-aa05-b64c70bdc8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.191397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d32d2-d84a-4243-ab24-11256ce91520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:20.192571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d32d2-d84a-4243-ab24-11256ce91520 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.193079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d32d2-d84a-4243-ab24-11256ce91520 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.193568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d32d2-d84a-4243-ab24-11256ce91520 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.194677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d32d2-d84a-4243-ab24-11256ce91520 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:20.195210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d32d2-d84a-4243-ab24-11256ce91520 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:20.195858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bc4e8b65-b3c4-4bb1-aa05-b64c70bdc8bc None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:20.197069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bc4e8b65-b3c4-4bb1-aa05-b64c70bdc8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:20.197431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d32d2-d84a-4243-ab24-11256ce91520 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:20.219477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 327/635] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 13 08:48:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:20.220771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d32d2-d84a-4243-ab24-11256ce91520 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:20.276632 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 309/636] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:20.277414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e8b65-b3c4-4bb1-aa05-b64c70bdc8bc None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:20.359564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d32d2-d84a-4243-ab24-11256ce91520 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:20.381064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:20.386960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:20.400510 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5d570-cd41-40d9-94fd-8ca428ebce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:20.401376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5d570-cd41-40d9-94fd-8ca428ebce87 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.401835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5d570-cd41-40d9-94fd-8ca428ebce87 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.402172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5d570-cd41-40d9-94fd-8ca428ebce87 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.408535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e8b65-b3c4-4bb1-aa05-b64c70bdc8bc None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:20.411309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5d570-cd41-40d9-94fd-8ca428ebce87 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:20.413697 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5d570-cd41-40d9-94fd-8ca428ebce87 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-79a5d570-cd41-40d9-94fd-8ca428ebce87 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:20.429856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:20.431080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:20.432278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:20.432278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:20.432278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:20.432278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:20.432278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:20.432278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:20.432278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:20.432278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:20.432278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:20.432278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:20.439008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 328/637] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 13 08:48:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:20.441333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5d570-cd41-40d9-94fd-8ca428ebce87 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:20.458332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a743d9-6791-46b9-afca-41dce3a523d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:20.459746 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a743d9-6791-46b9-afca-41dce3a523d8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.460470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a743d9-6791-46b9-afca-41dce3a523d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.460931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a743d9-6791-46b9-afca-41dce3a523d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.551694 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a743d9-6791-46b9-afca-41dce3a523d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:20.553371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a743d9-6791-46b9-afca-41dce3a523d8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:20.560518 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5d570-cd41-40d9-94fd-8ca428ebce87 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:20.600720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0168cd-f2b9-45a9-b402-c5d3488c0adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:20.601802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0168cd-f2b9-45a9-b402-c5d3488c0adb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.602247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0168cd-f2b9-45a9-b402-c5d3488c0adb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.602684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0168cd-f2b9-45a9-b402-c5d3488c0adb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.614257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0168cd-f2b9-45a9-b402-c5d3488c0adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-9a0168cd-f2b9-45a9-b402-c5d3488c0adb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:20.622407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:20.623497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:20.624399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:20.628037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 329/638] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:20.629056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0168cd-f2b9-45a9-b402-c5d3488c0adb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:20.650057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:20.665603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 310/639] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:20.670062 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a743d9-6791-46b9-afca-41dce3a523d8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:20.672566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:20.741913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0168cd-f2b9-45a9-b402-c5d3488c0adb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:20.778649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a743d9-6791-46b9-afca-41dce3a523d8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:20.780528 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ae375-89cd-482f-bcaa-5c1b6bb52a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:20.781204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae375-89cd-482f-bcaa-5c1b6bb52a74 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.781480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae375-89cd-482f-bcaa-5c1b6bb52a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.781758 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae375-89cd-482f-bcaa-5c1b6bb52a74 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.782629 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ae375-89cd-482f-bcaa-5c1b6bb52a74 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:20.782994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ae375-89cd-482f-bcaa-5c1b6bb52a74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:20.784580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ae375-89cd-482f-bcaa-5c1b6bb52a74 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:20.807097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 330/640] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 13 08:48:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:20.807097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ea5be-4b87-4503-907a-a754dc8786a9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.807097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ea5be-4b87-4503-907a-a754dc8786a9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.807097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ea5be-4b87-4503-907a-a754dc8786a9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.807097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 331/641] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:20.807856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ea5be-4b87-4503-907a-a754dc8786a9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:20.821874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b67cc-8b82-4f66-b111-22420c7e0af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:20.822575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b67cc-8b82-4f66-b111-22420c7e0af3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.822856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b67cc-8b82-4f66-b111-22420c7e0af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.823136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b67cc-8b82-4f66-b111-22420c7e0af3 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.823907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b67cc-8b82-4f66-b111-22420c7e0af3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:20.824216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b67cc-8b82-4f66-b111-22420c7e0af3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:20.825576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b67cc-8b82-4f66-b111-22420c7e0af3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:20.834736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 311/642] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:20.835541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b67cc-8b82-4f66-b111-22420c7e0af3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:20.888369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:20.923354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:20.931867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ea5be-4b87-4503-907a-a754dc8786a9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:20.942831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b67cc-8b82-4f66-b111-22420c7e0af3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:20.968040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afca9c-8ca8-412a-a42c-5777c637da7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:20.968824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afca9c-8ca8-412a-a42c-5777c637da7a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.969109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afca9c-8ca8-412a-a42c-5777c637da7a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.969383 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afca9c-8ca8-412a-a42c-5777c637da7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.979794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afca9c-8ca8-412a-a42c-5777c637da7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:20.980291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fb517-ed85-4894-91be-f04bd7c4a975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:20.980938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fb517-ed85-4894-91be-f04bd7c4a975 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:20.981312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fb517-ed85-4894-91be-f04bd7c4a975 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:20.981411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afca9c-8ca8-412a-a42c-5777c637da7a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:20.981595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fb517-ed85-4894-91be-f04bd7c4a975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a0afca9c-8ca8-412a-a42c-5777c637da7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:20.999765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:21.000922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:21.002504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:21.002504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:21.002504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:21.002504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:21.002504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 332/643] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:21.003209 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240970af-ab0f-4cad-bebb-653a8d7f2890 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:21.003498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240970af-ab0f-4cad-bebb-653a8d7f2890 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:21.003774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240970af-ab0f-4cad-bebb-653a8d7f2890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:21.020076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-240970af-ab0f-4cad-bebb-653a8d7f2890 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:21.021403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-240970af-ab0f-4cad-bebb-653a8d7f2890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:21.059487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fb517-ed85-4894-91be-f04bd7c4a975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:21.060946 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fb517-ed85-4894-91be-f04bd7c4a975 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:21.091293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 333/644] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:21.093067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240970af-ab0f-4cad-bebb-653a8d7f2890 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.114384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:21.170125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 312/645] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:21.171216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fb517-ed85-4894-91be-f04bd7c4a975 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.191391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:21.214173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240970af-ab0f-4cad-bebb-653a8d7f2890 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:21.263674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b64585-11fe-4e10-96ff-a21aa2b57471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:21.264574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b64585-11fe-4e10-96ff-a21aa2b57471 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:21.264858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b64585-11fe-4e10-96ff-a21aa2b57471 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:21.265136 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b64585-11fe-4e10-96ff-a21aa2b57471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:21.276552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b64585-11fe-4e10-96ff-a21aa2b57471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:21.278021 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b64585-11fe-4e10-96ff-a21aa2b57471 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:21.282906 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fb517-ed85-4894-91be-f04bd7c4a975 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f2b64585-11fe-4e10-96ff-a21aa2b57471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:21.296944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:21.297854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:21.299033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:21.299033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:21.299033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:21.299033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:21.299033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 334/646] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:21.299033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b64585-11fe-4e10-96ff-a21aa2b57471 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.322270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cff92-f28e-42d8-9854-c73c679a40c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:21.322935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cff92-f28e-42d8-9854-c73c679a40c7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:21.323218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cff92-f28e-42d8-9854-c73c679a40c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:21.323493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cff92-f28e-42d8-9854-c73c679a40c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:21.324547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cff92-f28e-42d8-9854-c73c679a40c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:21.324547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cff92-f28e-42d8-9854-c73c679a40c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:21.325879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cff92-f28e-42d8-9854-c73c679a40c7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:21.348568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 313/647] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:21.348568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cff92-f28e-42d8-9854-c73c679a40c7 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.357875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:21.413094 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b64585-11fe-4e10-96ff-a21aa2b57471 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:21.445028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:21.454279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c8e70-a00b-4c94-ae1b-aa27cd8e593f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:21.455786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c8e70-a00b-4c94-ae1b-aa27cd8e593f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:21.456110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c8e70-a00b-4c94-ae1b-aa27cd8e593f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:21.456356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c8e70-a00b-4c94-ae1b-aa27cd8e593f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:21.458892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cff92-f28e-42d8-9854-c73c679a40c7 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:21.467335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c8e70-a00b-4c94-ae1b-aa27cd8e593f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-af0c8e70-a00b-4c94-ae1b-aa27cd8e593f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:21.475408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:21.476401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:21.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 335/648] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:21.478167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c8e70-a00b-4c94-ae1b-aa27cd8e593f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.500047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b87a4f-b5ff-41a4-9cd0-d597a08d69b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:21.500967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b87a4f-b5ff-41a4-9cd0-d597a08d69b8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:21.501318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b87a4f-b5ff-41a4-9cd0-d597a08d69b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:21.502242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b87a4f-b5ff-41a4-9cd0-d597a08d69b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:21.514222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b87a4f-b5ff-41a4-9cd0-d597a08d69b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:21.515665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b87a4f-b5ff-41a4-9cd0-d597a08d69b8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b7b87a4f-b5ff-41a4-9cd0-d597a08d69b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:21.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:21.535204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:21.536389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:21.536389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:21.536389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:21.536389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:21.536389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 314/649] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:21.536713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b87a4f-b5ff-41a4-9cd0-d597a08d69b8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.589886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c8e70-a00b-4c94-ae1b-aa27cd8e593f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:21.599311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:21.623098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d6c1f-3a67-43c0-8bfd-44d81aca7088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:21.623761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d6c1f-3a67-43c0-8bfd-44d81aca7088 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:21.624046 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d6c1f-3a67-43c0-8bfd-44d81aca7088 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:21.624324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d6c1f-3a67-43c0-8bfd-44d81aca7088 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:21.625043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d6c1f-3a67-43c0-8bfd-44d81aca7088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:21.625352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d6c1f-3a67-43c0-8bfd-44d81aca7088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:21.627423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d6c1f-3a67-43c0-8bfd-44d81aca7088 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:21.639150 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 336/650] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:21.639939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d6c1f-3a67-43c0-8bfd-44d81aca7088 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.646999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b87a4f-b5ff-41a4-9cd0-d597a08d69b8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:21.685983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fc19e-912b-40d5-bcfc-4a1726d9e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:21.687123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fc19e-912b-40d5-bcfc-4a1726d9e032 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:21.687619 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fc19e-912b-40d5-bcfc-4a1726d9e032 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:21.687906 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fc19e-912b-40d5-bcfc-4a1726d9e032 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:21.696207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fc19e-912b-40d5-bcfc-4a1726d9e032 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:21.697625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fc19e-912b-40d5-bcfc-4a1726d9e032 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f15fc19e-912b-40d5-bcfc-4a1726d9e032 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:21.713104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:21.714143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:21.715768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:21.715768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:21.715768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:21.715768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:21.715768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:21.715768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:21.715768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:21.715768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:21.715768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 315/651] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:21.715768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fc19e-912b-40d5-bcfc-4a1726d9e032 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.741807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d6c1f-3a67-43c0-8bfd-44d81aca7088 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:21.761594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:21.779287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98565950-4abd-4d8c-ac9f-c485e7edb195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:21.779959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98565950-4abd-4d8c-ac9f-c485e7edb195 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:21.780259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98565950-4abd-4d8c-ac9f-c485e7edb195 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:21.780571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98565950-4abd-4d8c-ac9f-c485e7edb195 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:21.781389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98565950-4abd-4d8c-ac9f-c485e7edb195 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:21.781753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98565950-4abd-4d8c-ac9f-c485e7edb195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:21.783158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98565950-4abd-4d8c-ac9f-c485e7edb195 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:21.796211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 337/652] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:21.798148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98565950-4abd-4d8c-ac9f-c485e7edb195 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.821564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fc19e-912b-40d5-bcfc-4a1726d9e032 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:21.845012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:21.859939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589d7ab-8037-4216-8113-10fae6e2a906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:21.861073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589d7ab-8037-4216-8113-10fae6e2a906 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:21.861073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589d7ab-8037-4216-8113-10fae6e2a906 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:21.861324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589d7ab-8037-4216-8113-10fae6e2a906 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:21.862066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589d7ab-8037-4216-8113-10fae6e2a906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:21.862375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589d7ab-8037-4216-8113-10fae6e2a906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:21.863758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589d7ab-8037-4216-8113-10fae6e2a906 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:21.875517 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 316/653] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:21.876390 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589d7ab-8037-4216-8113-10fae6e2a906 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.910814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98565950-4abd-4d8c-ac9f-c485e7edb195 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:21.923057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:21.948272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f873a-5676-47c4-8646-1efa9fdbbff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:21.949152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f873a-5676-47c4-8646-1efa9fdbbff1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:21.949576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f873a-5676-47c4-8646-1efa9fdbbff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:21.949896 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f873a-5676-47c4-8646-1efa9fdbbff1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:21.958593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f873a-5676-47c4-8646-1efa9fdbbff1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:21.960648 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f873a-5676-47c4-8646-1efa9fdbbff1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4f2f873a-5676-47c4-8646-1efa9fdbbff1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:21.976575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:21.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:21.978537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 338/654] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:21.979156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f873a-5676-47c4-8646-1efa9fdbbff1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:21.986435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589d7ab-8037-4216-8113-10fae6e2a906 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:22.023869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c2196-411d-4a2e-8646-20e929bbca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:22.024535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c2196-411d-4a2e-8646-20e929bbca4d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.024932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c2196-411d-4a2e-8646-20e929bbca4d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.025123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c2196-411d-4a2e-8646-20e929bbca4d None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.025867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c2196-411d-4a2e-8646-20e929bbca4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:22.026208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c2196-411d-4a2e-8646-20e929bbca4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:22.027645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c2196-411d-4a2e-8646-20e929bbca4d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:22.041077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 317/655] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:22.041878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c2196-411d-4a2e-8646-20e929bbca4d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:22.089279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:22.091812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f873a-5676-47c4-8646-1efa9fdbbff1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:22.132199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eca1ea-82bb-44b8-853e-e63b8a589fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:22.133099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eca1ea-82bb-44b8-853e-e63b8a589fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.133390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eca1ea-82bb-44b8-853e-e63b8a589fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.133660 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eca1ea-82bb-44b8-853e-e63b8a589fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.146672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eca1ea-82bb-44b8-853e-e63b8a589fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:22.148266 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eca1ea-82bb-44b8-853e-e63b8a589fc0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:22.151220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c2196-411d-4a2e-8646-20e929bbca4d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-54eca1ea-82bb-44b8-853e-e63b8a589fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:22.166263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:22.167229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:22.168123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:22.168123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:22.168123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:22.168123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:22.168123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 339/656] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:22.168407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eca1ea-82bb-44b8-853e-e63b8a589fc0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:22.193586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33183f-6e0a-4ac5-8d0d-ad49cc8fd40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:22.194421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33183f-6e0a-4ac5-8d0d-ad49cc8fd40d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.194554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:22.194695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33183f-6e0a-4ac5-8d0d-ad49cc8fd40d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.194972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33183f-6e0a-4ac5-8d0d-ad49cc8fd40d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.209653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33183f-6e0a-4ac5-8d0d-ad49cc8fd40d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:22.209653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33183f-6e0a-4ac5-8d0d-ad49cc8fd40d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ee33183f-6e0a-4ac5-8d0d-ad49cc8fd40d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:22.216765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:22.217701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:22.218846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:22.218846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:22.218846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:22.218846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:22.218846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:22.218846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:22.218846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:22.218846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:22.218846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 318/657] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:22.218846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33183f-6e0a-4ac5-8d0d-ad49cc8fd40d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:22.270879 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eca1ea-82bb-44b8-853e-e63b8a589fc0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:22.303207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7188e30-a0b1-407b-8ad3-cbb54bf2dac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:22.303869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7188e30-a0b1-407b-8ad3-cbb54bf2dac1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.304148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7188e30-a0b1-407b-8ad3-cbb54bf2dac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.304419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7188e30-a0b1-407b-8ad3-cbb54bf2dac1 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.305246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7188e30-a0b1-407b-8ad3-cbb54bf2dac1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:22.305548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7188e30-a0b1-407b-8ad3-cbb54bf2dac1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:22.307010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7188e30-a0b1-407b-8ad3-cbb54bf2dac1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:22.316484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:22.319391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33183f-6e0a-4ac5-8d0d-ad49cc8fd40d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:22.324605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 340/658] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 13 08:48:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:22.328291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835d532-bdb6-48e7-82f4-e7b8d607ab73 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.328291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835d532-bdb6-48e7-82f4-e7b8d607ab73 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.328291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835d532-bdb6-48e7-82f4-e7b8d607ab73 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.328291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 341/659] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:22.328291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835d532-bdb6-48e7-82f4-e7b8d607ab73 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:22.355481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f630a9-c8e9-41db-8de5-e4d7305e0b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:22.356132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f630a9-c8e9-41db-8de5-e4d7305e0b44 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.356419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f630a9-c8e9-41db-8de5-e4d7305e0b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.356694 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f630a9-c8e9-41db-8de5-e4d7305e0b44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.357510 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f630a9-c8e9-41db-8de5-e4d7305e0b44 None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=cf67fa55f4d64b7ebeeb1490453661e0) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:22.357840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f630a9-c8e9-41db-8de5-e4d7305e0b44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:22.359291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f630a9-c8e9-41db-8de5-e4d7305e0b44 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:22.365463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 319/660] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:22 2022] GET /identity/v3/role_assignments?user.id=cf67fa55f4d64b7ebeeb1490453661e0&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:22.366259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f630a9-c8e9-41db-8de5-e4d7305e0b44 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:22.429439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:22.452712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835d532-bdb6-48e7-82f4-e7b8d607ab73 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:22.475936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f630a9-c8e9-41db-8de5-e4d7305e0b44 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:22.491885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec0f91-a31f-4ad7-be43-9c9c00e40bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:22.492552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec0f91-a31f-4ad7-be43-9c9c00e40bff None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.492829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec0f91-a31f-4ad7-be43-9c9c00e40bff None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.493097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec0f91-a31f-4ad7-be43-9c9c00e40bff None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.493822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec0f91-a31f-4ad7-be43-9c9c00e40bff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:22.494152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec0f91-a31f-4ad7-be43-9c9c00e40bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:22.495548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec0f91-a31f-4ad7-be43-9c9c00e40bff None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:22.508804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 342/661] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:22.513922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec0f91-a31f-4ad7-be43-9c9c00e40bff None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:22.514096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4615e-eb1b-48f8-b970-9e1853d2084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:22.514667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4615e-eb1b-48f8-b970-9e1853d2084f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.514942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4615e-eb1b-48f8-b970-9e1853d2084f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.515217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4615e-eb1b-48f8-b970-9e1853d2084f None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.516114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4615e-eb1b-48f8-b970-9e1853d2084f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:22.516422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4615e-eb1b-48f8-b970-9e1853d2084f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:22.518188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4615e-eb1b-48f8-b970-9e1853d2084f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:22.534493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 320/662] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:22.535613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e866c58-f1ce-4aef-b986-46d0f25723f7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.535903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e866c58-f1ce-4aef-b986-46d0f25723f7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.536180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e866c58-f1ce-4aef-b986-46d0f25723f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.555436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5e866c58-f1ce-4aef-b986-46d0f25723f7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:22.556627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5e866c58-f1ce-4aef-b986-46d0f25723f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:22.576386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:22.631770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 321/663] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:22.632804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec0f91-a31f-4ad7-be43-9c9c00e40bff None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:22.635963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e866c58-f1ce-4aef-b986-46d0f25723f7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:22.672269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de350326-9f0c-4936-92c1-d515400a8bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:22.672917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de350326-9f0c-4936-92c1-d515400a8bdb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.673192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de350326-9f0c-4936-92c1-d515400a8bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.673525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de350326-9f0c-4936-92c1-d515400a8bdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.686142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:22.747333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de350326-9f0c-4936-92c1-d515400a8bdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:22.748713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de350326-9f0c-4936-92c1-d515400a8bdb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:22.755484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e866c58-f1ce-4aef-b986-46d0f25723f7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:22.791688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9004e-fdcb-4cc4-8440-8daebb7a7376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:22.792489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9004e-fdcb-4cc4-8440-8daebb7a7376 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.792764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9004e-fdcb-4cc4-8440-8daebb7a7376 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.793034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9004e-fdcb-4cc4-8440-8daebb7a7376 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.800896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9004e-fdcb-4cc4-8440-8daebb7a7376 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:22.802354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9004e-fdcb-4cc4-8440-8daebb7a7376 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3fb9004e-fdcb-4cc4-8440-8daebb7a7376 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:22.817615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:22.818635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 322/664] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:22.819572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9004e-fdcb-4cc4-8440-8daebb7a7376 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:22.839792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:22.865299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 343/665] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:22.866059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de350326-9f0c-4936-92c1-d515400a8bdb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:22.914008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:22.928883 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9004e-fdcb-4cc4-8440-8daebb7a7376 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:22.965233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7142a11-45f4-4583-ac09-817bdfbf82ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:22.966097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7142a11-45f4-4583-ac09-817bdfbf82ae None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:22.966393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7142a11-45f4-4583-ac09-817bdfbf82ae None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:22.966668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7142a11-45f4-4583-ac09-817bdfbf82ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:22.967498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7142a11-45f4-4583-ac09-817bdfbf82ae None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=8af26e036f4e466ebb923771c1f5f502) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:22.967812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7142a11-45f4-4583-ac09-817bdfbf82ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:22.969244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7142a11-45f4-4583-ac09-817bdfbf82ae None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:22.975669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 323/666] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:22 2022] GET /identity/v3/role_assignments?user.id=8af26e036f4e466ebb923771c1f5f502&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:22.976284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7142a11-45f4-4583-ac09-817bdfbf82ae None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:22.986458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de350326-9f0c-4936-92c1-d515400a8bdb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:23.026602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366400c-da15-441d-8196-d6ca738716de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:23.027319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366400c-da15-441d-8196-d6ca738716de None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:23.027606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366400c-da15-441d-8196-d6ca738716de None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:23.027883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366400c-da15-441d-8196-d6ca738716de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:23.040604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366400c-da15-441d-8196-d6ca738716de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f366400c-da15-441d-8196-d6ca738716de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:23.049414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:23.050498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:23.051338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 344/667] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:23.052027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366400c-da15-441d-8196-d6ca738716de None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:23.097516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:23.125940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7142a11-45f4-4583-ac09-817bdfbf82ae None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:23.174599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:23.178056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680e8ae-3948-4494-a947-7e2e2f2580e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:23.178056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680e8ae-3948-4494-a947-7e2e2f2580e3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:23.178056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680e8ae-3948-4494-a947-7e2e2f2580e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:23.178056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680e8ae-3948-4494-a947-7e2e2f2580e3 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:23.178735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366400c-da15-441d-8196-d6ca738716de None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:23.178823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680e8ae-3948-4494-a947-7e2e2f2580e3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:23.179061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680e8ae-3948-4494-a947-7e2e2f2580e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:23.181571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680e8ae-3948-4494-a947-7e2e2f2580e3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:23.203976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 324/668] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Jul 13 08:48:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:23.204702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680e8ae-3948-4494-a947-7e2e2f2580e3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:23.225558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807eaa1-3300-4104-b580-93dc30b73bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:23.226387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807eaa1-3300-4104-b580-93dc30b73bcb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:23.226664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807eaa1-3300-4104-b580-93dc30b73bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:23.226934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807eaa1-3300-4104-b580-93dc30b73bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:23.242813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807eaa1-3300-4104-b580-93dc30b73bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:23.244540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807eaa1-3300-4104-b580-93dc30b73bcb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6807eaa1-3300-4104-b580-93dc30b73bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:23.263723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:23.264652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:23.265559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:23.265559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:23.265559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:23.265559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:23.267877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 345/669] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:23.268984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807eaa1-3300-4104-b580-93dc30b73bcb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:23.331604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680e8ae-3948-4494-a947-7e2e2f2580e3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:23.369792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484ef48-dc7b-46a1-bd59-ac0ab1926793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:23.370848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ef48-dc7b-46a1-bd59-ac0ab1926793 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:23.371428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ef48-dc7b-46a1-bd59-ac0ab1926793 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:23.371900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ef48-dc7b-46a1-bd59-ac0ab1926793 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:23.372830 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484ef48-dc7b-46a1-bd59-ac0ab1926793 None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:23.373328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484ef48-dc7b-46a1-bd59-ac0ab1926793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:23.374931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484ef48-dc7b-46a1-bd59-ac0ab1926793 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:23.378534 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:23.385992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 325/670] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:23.391157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484ef48-dc7b-46a1-bd59-ac0ab1926793 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:23.411463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807eaa1-3300-4104-b580-93dc30b73bcb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:23.456346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c0821-1061-4312-a277-d11c1cebd5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:23.457042 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c0821-1061-4312-a277-d11c1cebd5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:23.457283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c0821-1061-4312-a277-d11c1cebd5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:23.457563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c0821-1061-4312-a277-d11c1cebd5e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:23.483651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:23.501364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484ef48-dc7b-46a1-bd59-ac0ab1926793 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:23.534706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c0821-1061-4312-a277-d11c1cebd5e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:23.536462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c0821-1061-4312-a277-d11c1cebd5e0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:23.543530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5702f-8b3c-4bea-aedd-e332332a8db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:23.544208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5702f-8b3c-4bea-aedd-e332332a8db7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:23.544492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5702f-8b3c-4bea-aedd-e332332a8db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:23.544769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5702f-8b3c-4bea-aedd-e332332a8db7 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:23.546187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5702f-8b3c-4bea-aedd-e332332a8db7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:23.546669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5702f-8b3c-4bea-aedd-e332332a8db7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:23.548056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5702f-8b3c-4bea-aedd-e332332a8db7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:23.569551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 326/671] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:23.569551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5702f-8b3c-4bea-aedd-e332332a8db7 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:23.638213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:23.659100 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 346/672] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:23.660343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c0821-1061-4312-a277-d11c1cebd5e0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:23.690872 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5702f-8b3c-4bea-aedd-e332332a8db7 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:23.715725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:23.733740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e585e-ad48-4837-be6c-4ebc97176d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:23.734572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e585e-ad48-4837-be6c-4ebc97176d6f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:23.734858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e585e-ad48-4837-be6c-4ebc97176d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:23.735175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e585e-ad48-4837-be6c-4ebc97176d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:23.748884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e585e-ad48-4837-be6c-4ebc97176d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:23.750729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e585e-ad48-4837-be6c-4ebc97176d6f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:23.763291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c0821-1061-4312-a277-d11c1cebd5e0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-282e585e-ad48-4837-be6c-4ebc97176d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:23.775907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:23.776817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:23.777800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:23.777800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:23.777800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:23.777800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:23.777800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 327/673] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:23.778465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e585e-ad48-4837-be6c-4ebc97176d6f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:23.805563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05cb9c-1961-4b2b-8262-29f068ccd0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:23.806485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05cb9c-1961-4b2b-8262-29f068ccd0ad None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:23.806880 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05cb9c-1961-4b2b-8262-29f068ccd0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:23.807253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05cb9c-1961-4b2b-8262-29f068ccd0ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:23.815404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05cb9c-1961-4b2b-8262-29f068ccd0ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:23.816971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05cb9c-1961-4b2b-8262-29f068ccd0ad None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-9b05cb9c-1961-4b2b-8262-29f068ccd0ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:23.831546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:23.832431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:23.833321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:23.833321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:23.833321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:23.833321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:23.833321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:23.833321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:23.833321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:23.833321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:23.837588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 347/674] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:23.838501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05cb9c-1961-4b2b-8262-29f068ccd0ad None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:23.912950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e585e-ad48-4837-be6c-4ebc97176d6f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:23.927450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:23.948910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05cb9c-1961-4b2b-8262-29f068ccd0ad None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:23.963016 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01397108-d970-4798-8bec-04766036db07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:23.963820 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01397108-d970-4798-8bec-04766036db07 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:23.964105 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01397108-d970-4798-8bec-04766036db07 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:23.964381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01397108-d970-4798-8bec-04766036db07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:23.966887 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:23.982055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01397108-d970-4798-8bec-04766036db07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:23.984187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01397108-d970-4798-8bec-04766036db07 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:23.989575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c936e-ca7b-48d3-a1b2-edeee9d71b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:23.990302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c936e-ca7b-48d3-a1b2-edeee9d71b27 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:23.990586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c936e-ca7b-48d3-a1b2-edeee9d71b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:23.990863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c936e-ca7b-48d3-a1b2-edeee9d71b27 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:23.991616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c936e-ca7b-48d3-a1b2-edeee9d71b27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:23.991928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c936e-ca7b-48d3-a1b2-edeee9d71b27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:23.993287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c936e-ca7b-48d3-a1b2-edeee9d71b27 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-01397108-d970-4798-8bec-04766036db07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:24.008223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:24.009132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:24.010019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:24.010019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:24.010019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:24.010019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:24.010719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 348/675] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:24.011207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c936e-ca7b-48d3-a1b2-edeee9d71b27 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:24.017755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 328/676] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:24.020337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01397108-d970-4798-8bec-04766036db07 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:24.139536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01397108-d970-4798-8bec-04766036db07 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:24.145591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c936e-ca7b-48d3-a1b2-edeee9d71b27 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:24.183561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13790336-900a-48fa-a4cf-5d6c9a88ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:24.184318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13790336-900a-48fa-a4cf-5d6c9a88ce1a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.184765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13790336-900a-48fa-a4cf-5d6c9a88ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.185137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13790336-900a-48fa-a4cf-5d6c9a88ce1a None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:24.186200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13790336-900a-48fa-a4cf-5d6c9a88ce1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:24.186579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13790336-900a-48fa-a4cf-5d6c9a88ce1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:24.186867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f7af1-0447-4083-9af7-6d0d27326ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:24.188185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f7af1-0447-4083-9af7-6d0d27326ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.188290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13790336-900a-48fa-a4cf-5d6c9a88ce1a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:24.188737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f7af1-0447-4083-9af7-6d0d27326ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.190209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f7af1-0447-4083-9af7-6d0d27326ff9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:24.200612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f7af1-0447-4083-9af7-6d0d27326ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:24.201867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 349/677] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:24.202853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f7af1-0447-4083-9af7-6d0d27326ff9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:24.206905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13790336-900a-48fa-a4cf-5d6c9a88ce1a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d93f7af1-0447-4083-9af7-6d0d27326ff9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:24.227077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:24.228069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:24.229093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:24.229093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:24.229093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:24.229093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:24.229093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:24.229093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:24.229093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:24.229093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:24.229093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:24.229093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:24.232015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 329/678] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:24.232986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f7af1-0447-4083-9af7-6d0d27326ff9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:24.246412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:24.266427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:24.324580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13790336-900a-48fa-a4cf-5d6c9a88ce1a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:24.351353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f7af1-0447-4083-9af7-6d0d27326ff9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:24.366590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf9209-e73b-4745-b319-23cf68238c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:24.367560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9209-e73b-4745-b319-23cf68238c64 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.368094 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9209-e73b-4745-b319-23cf68238c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.368694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9209-e73b-4745-b319-23cf68238c64 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:24.369617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf9209-e73b-4745-b319-23cf68238c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:24.370156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf9209-e73b-4745-b319-23cf68238c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:24.371907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf9209-e73b-4745-b319-23cf68238c64 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:24.386591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 350/679] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:24.387806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf9209-e73b-4745-b319-23cf68238c64 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:24.401522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f5a3a-ef59-47be-8894-02b647ff306a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:24.402675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5a3a-ef59-47be-8894-02b647ff306a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.403163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5a3a-ef59-47be-8894-02b647ff306a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.403739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5a3a-ef59-47be-8894-02b647ff306a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:24.404781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f5a3a-ef59-47be-8894-02b647ff306a None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=b3f12750f91645748b37e2b159a9bd35) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:24.405295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f5a3a-ef59-47be-8894-02b647ff306a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:24.407141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f5a3a-ef59-47be-8894-02b647ff306a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:24.422370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 330/680] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:24 2022] GET /identity/v3/role_assignments?user.id=b3f12750f91645748b37e2b159a9bd35&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:24.423910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f5a3a-ef59-47be-8894-02b647ff306a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:24.527114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:24.551384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f5a3a-ef59-47be-8894-02b647ff306a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:24.575760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:24.588627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf9209-e73b-4745-b319-23cf68238c64 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:24.596702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153dc3e-04f5-4eb5-af30-c1dfa28d89a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:24.597499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153dc3e-04f5-4eb5-af30-c1dfa28d89a2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.597839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153dc3e-04f5-4eb5-af30-c1dfa28d89a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.598141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153dc3e-04f5-4eb5-af30-c1dfa28d89a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:24.607522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153dc3e-04f5-4eb5-af30-c1dfa28d89a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:24.608958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153dc3e-04f5-4eb5-af30-c1dfa28d89a2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:24.630355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f12a3-4214-4f12-a3a4-6868703959ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7153dc3e-04f5-4eb5-af30-c1dfa28d89a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:24.630572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:24.631409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:24.632398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:24.632398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:24.632398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:24.632398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:24.632398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:24.632398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:24.632398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:24.632398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:24.632845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f12a3-4214-4f12-a3a4-6868703959ba None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.632845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f12a3-4214-4f12-a3a4-6868703959ba None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.632845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f12a3-4214-4f12-a3a4-6868703959ba None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:24.633049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 331/681] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:24.633653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f12a3-4214-4f12-a3a4-6868703959ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:24.633653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f12a3-4214-4f12-a3a4-6868703959ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:24.635099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f12a3-4214-4f12-a3a4-6868703959ba None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:24.637414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0782d-2775-4014-977d-5e6f940f6237 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.637696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0782d-2775-4014-977d-5e6f940f6237 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.637972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0782d-2775-4014-977d-5e6f940f6237 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:24.639317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 332/682] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:24.640535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0782d-2775-4014-977d-5e6f940f6237 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:24.656939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 351/683] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 13 08:48:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:24.659155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b22bd-11b8-492f-a750-d861e0d27d59 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.659647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b22bd-11b8-492f-a750-d861e0d27d59 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.660138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b22bd-11b8-492f-a750-d861e0d27d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:24.680953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-906b22bd-11b8-492f-a750-d861e0d27d59 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:24.682820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-906b22bd-11b8-492f-a750-d861e0d27d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:24.756341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0782d-2775-4014-977d-5e6f940f6237 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:24.772974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 352/684] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:24.774376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b22bd-11b8-492f-a750-d861e0d27d59 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:24.808029 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80d0b5-9fff-4451-87f5-353410833564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:24.808652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d0b5-9fff-4451-87f5-353410833564 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.808938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d0b5-9fff-4451-87f5-353410833564 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.809267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d0b5-9fff-4451-87f5-353410833564 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:24.810091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80d0b5-9fff-4451-87f5-353410833564 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:24.810412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80d0b5-9fff-4451-87f5-353410833564 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:24.811927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80d0b5-9fff-4451-87f5-353410833564 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:24.825055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 333/685] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:24.825894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80d0b5-9fff-4451-87f5-353410833564 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:24.836894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:24.879168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:24.905468 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b22bd-11b8-492f-a750-d861e0d27d59 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:24.942752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80d0b5-9fff-4451-87f5-353410833564 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:24.951515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90bc5b-6912-4f58-84e7-87b79ecbf404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:24.952166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90bc5b-6912-4f58-84e7-87b79ecbf404 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.952449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90bc5b-6912-4f58-84e7-87b79ecbf404 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.952747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90bc5b-6912-4f58-84e7-87b79ecbf404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:24.990514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1eeedf-c05a-4584-aa4a-c768bd2bb1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:24.991468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1eeedf-c05a-4584-aa4a-c768bd2bb1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:24.991754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1eeedf-c05a-4584-aa4a-c768bd2bb1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:24.992029 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1eeedf-c05a-4584-aa4a-c768bd2bb1f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:25.006465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1eeedf-c05a-4584-aa4a-c768bd2bb1f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:25.008085 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1eeedf-c05a-4584-aa4a-c768bd2bb1f1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7f1eeedf-c05a-4584-aa4a-c768bd2bb1f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:25.029896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:25.031358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:25.034137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:25.034137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:25.034137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:25.034137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:25.034137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 334/686] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:25.034137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1eeedf-c05a-4584-aa4a-c768bd2bb1f1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:25.038671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90bc5b-6912-4f58-84e7-87b79ecbf404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:25.040144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90bc5b-6912-4f58-84e7-87b79ecbf404 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:25.106489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:25.122421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:25.148169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 353/687] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:25.149162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90bc5b-6912-4f58-84e7-87b79ecbf404 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:25.152708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1eeedf-c05a-4584-aa4a-c768bd2bb1f1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:25.189955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b5f3a-f781-421f-88f2-e8740c6523a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:25.190621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b5f3a-f781-421f-88f2-e8740c6523a0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:25.190905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b5f3a-f781-421f-88f2-e8740c6523a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:25.191185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b5f3a-f781-421f-88f2-e8740c6523a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:25.192023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b5f3a-f781-421f-88f2-e8740c6523a0 None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=12824152139841138f457dfe5c9758e5) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:25.192327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b5f3a-f781-421f-88f2-e8740c6523a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:25.193764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b5f3a-f781-421f-88f2-e8740c6523a0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:25.200308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 335/688] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:25 2022] GET /identity/v3/role_assignments?user.id=12824152139841138f457dfe5c9758e5&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:25.201120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b5f3a-f781-421f-88f2-e8740c6523a0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:25.262156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90bc5b-6912-4f58-84e7-87b79ecbf404 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:25.301582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0ec2e-226c-4153-87da-b1d345212d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:25.302330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0ec2e-226c-4153-87da-b1d345212d6a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:25.302621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0ec2e-226c-4153-87da-b1d345212d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:25.302905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0ec2e-226c-4153-87da-b1d345212d6a None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:25.303661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0ec2e-226c-4153-87da-b1d345212d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:25.303976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0ec2e-226c-4153-87da-b1d345212d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:25.305866 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0ec2e-226c-4153-87da-b1d345212d6a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:25.320175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 354/689] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:25.326327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a2a13-27a0-4cc0-be8b-90e7aa190ab6 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:25.326628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a2a13-27a0-4cc0-be8b-90e7aa190ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:25.326901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a2a13-27a0-4cc0-be8b-90e7aa190ab6 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:25.328794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 355/690] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:25.330264 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a2a13-27a0-4cc0-be8b-90e7aa190ab6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:25.354669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b5f3a-f781-421f-88f2-e8740c6523a0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:25.380543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:25.382606 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:25.395691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88343c2d-6f67-4d9d-8ede-77917e2675cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:25.396806 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88343c2d-6f67-4d9d-8ede-77917e2675cc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:25.397238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88343c2d-6f67-4d9d-8ede-77917e2675cc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:25.397659 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88343c2d-6f67-4d9d-8ede-77917e2675cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:25.410067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88343c2d-6f67-4d9d-8ede-77917e2675cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:25.411839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88343c2d-6f67-4d9d-8ede-77917e2675cc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-88343c2d-6f67-4d9d-8ede-77917e2675cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:25.431584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:25.432519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:25.433557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:25.433557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:25.433557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:25.433557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:25.435848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 336/691] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:25.437251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb1942-e613-4c51-a8db-e7ea64403ab9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:25.437678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb1942-e613-4c51-a8db-e7ea64403ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:25.438139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb1942-e613-4c51-a8db-e7ea64403ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:25.449027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a2a13-27a0-4cc0-be8b-90e7aa190ab6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:25.456779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-efcb1942-e613-4c51-a8db-e7ea64403ab9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:25.457980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-efcb1942-e613-4c51-a8db-e7ea64403ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:25.489138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c98991-db6a-46bc-9974-af3e59a2d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:25.489818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c98991-db6a-46bc-9974-af3e59a2d8e2 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:25.490817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c98991-db6a-46bc-9974-af3e59a2d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:25.490817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c98991-db6a-46bc-9974-af3e59a2d8e2 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:25.491224 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c98991-db6a-46bc-9974-af3e59a2d8e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:25.492668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c98991-db6a-46bc-9974-af3e59a2d8e2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:25.540143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 337/692] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:25.540942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb1942-e613-4c51-a8db-e7ea64403ab9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:25.608219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 356/693] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:48:25 2022] POST /identity/v3/users => generated 254 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:25.609799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c98991-db6a-46bc-9974-af3e59a2d8e2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:25.664480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb1942-e613-4c51-a8db-e7ea64403ab9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:25.668526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:25.710539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4b4b0-8f28-4d74-b6b3-c4da9c8caf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:25.711558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4b4b0-8f28-4d74-b6b3-c4da9c8caf01 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:25.712066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4b4b0-8f28-4d74-b6b3-c4da9c8caf01 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:25.712523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4b4b0-8f28-4d74-b6b3-c4da9c8caf01 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:25.713469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4b4b0-8f28-4d74-b6b3-c4da9c8caf01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:25.713960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4b4b0-8f28-4d74-b6b3-c4da9c8caf01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:25.716185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4b4b0-8f28-4d74-b6b3-c4da9c8caf01 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:25.732444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 338/694] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:25.739135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4b4b0-8f28-4d74-b6b3-c4da9c8caf01 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:25.839504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:25.858366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c98991-db6a-46bc-9974-af3e59a2d8e2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:25.861509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4b4b0-8f28-4d74-b6b3-c4da9c8caf01 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:25.908070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6818b5-1abb-40ca-8fa9-0070d9fae86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:25.909490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6818b5-1abb-40ca-8fa9-0070d9fae86e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:25.910000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6818b5-1abb-40ca-8fa9-0070d9fae86e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:25.910479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6818b5-1abb-40ca-8fa9-0070d9fae86e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:25.915204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805e160-2e76-40c8-abb2-473ba22581d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:25.915937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805e160-2e76-40c8-abb2-473ba22581d9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:25.916214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805e160-2e76-40c8-abb2-473ba22581d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:25.916480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805e160-2e76-40c8-abb2-473ba22581d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:25.920378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6818b5-1abb-40ca-8fa9-0070d9fae86e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:25.922358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6818b5-1abb-40ca-8fa9-0070d9fae86e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:25.928937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805e160-2e76-40c8-abb2-473ba22581d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e805e160-2e76-40c8-abb2-473ba22581d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:25.939681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:25.940651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:25.941495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:25.942662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 357/695] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-9e6818b5-1abb-40ca-8fa9-0070d9fae86e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:25.943351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:25.944401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:25.945487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:25.945487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:25.945487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:25.945487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:25.945487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:25.945487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:25.945487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:25.945487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:25.947552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805e160-2e76-40c8-abb2-473ba22581d9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:25.947711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 339/696] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:25.955723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6818b5-1abb-40ca-8fa9-0070d9fae86e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:25.994092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:26.081859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805e160-2e76-40c8-abb2-473ba22581d9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:26.089001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6818b5-1abb-40ca-8fa9-0070d9fae86e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:26.132677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de59b25-f8e1-4e27-8c2b-e6d3b42c051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:26.133709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de59b25-f8e1-4e27-8c2b-e6d3b42c051a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:26.134157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de59b25-f8e1-4e27-8c2b-e6d3b42c051a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:26.134442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de59b25-f8e1-4e27-8c2b-e6d3b42c051a None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:26.135460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de59b25-f8e1-4e27-8c2b-e6d3b42c051a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:26.135864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de59b25-f8e1-4e27-8c2b-e6d3b42c051a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:26.137415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de59b25-f8e1-4e27-8c2b-e6d3b42c051a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:26.138708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5854c-b5c3-4cdd-8276-2b5c58096184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:26.139487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5854c-b5c3-4cdd-8276-2b5c58096184 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:26.139769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5854c-b5c3-4cdd-8276-2b5c58096184 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:26.140041 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5854c-b5c3-4cdd-8276-2b5c58096184 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:26.140894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5854c-b5c3-4cdd-8276-2b5c58096184 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:26.141199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5854c-b5c3-4cdd-8276-2b5c58096184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:26.142948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5854c-b5c3-4cdd-8276-2b5c58096184 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:26.151058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 340/697] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:26.165088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de59b25-f8e1-4e27-8c2b-e6d3b42c051a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:26.166680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 358/698] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:26.178064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5854c-b5c3-4cdd-8276-2b5c58096184 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:26.222499 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:26.292772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:26.299358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de59b25-f8e1-4e27-8c2b-e6d3b42c051a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:26.413042 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5854c-b5c3-4cdd-8276-2b5c58096184 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:26.463489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcf1b2-6580-4603-b730-67a572f4c038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:26.464068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf1b2-6580-4603-b730-67a572f4c038 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:26.464348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf1b2-6580-4603-b730-67a572f4c038 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:26.465178 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcf1b2-6580-4603-b730-67a572f4c038 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:26.466016 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcf1b2-6580-4603-b730-67a572f4c038 None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=906fc971a5ee4ffd9503ab13d46cd605) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:26.466453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcf1b2-6580-4603-b730-67a572f4c038 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:26.467894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcf1b2-6580-4603-b730-67a572f4c038 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:26.475010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 341/699] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:26 2022] GET /identity/v3/role_assignments?user.id=906fc971a5ee4ffd9503ab13d46cd605&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:26.513872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0338a-dc2b-41d4-99f3-7fe52582c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:26.515082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0338a-dc2b-41d4-99f3-7fe52582c211 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:26.515638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0338a-dc2b-41d4-99f3-7fe52582c211 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:26.516166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0338a-dc2b-41d4-99f3-7fe52582c211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:26.537730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0338a-dc2b-41d4-99f3-7fe52582c211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:26.539625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0338a-dc2b-41d4-99f3-7fe52582c211 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-23c0338a-dc2b-41d4-99f3-7fe52582c211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:26.567589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:26.568535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:26.569432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:26.569432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:26.569432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:26.569432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:26.571592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 359/700] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:26.577583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcf1b2-6580-4603-b730-67a572f4c038 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:26.703561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcf1b2-6580-4603-b730-67a572f4c038 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:26.739120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f1546-3007-4af2-a237-075ed8dce781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:26.739709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f1546-3007-4af2-a237-075ed8dce781 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:26.739988 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f1546-3007-4af2-a237-075ed8dce781 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:26.740259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f1546-3007-4af2-a237-075ed8dce781 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:26.740969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f1546-3007-4af2-a237-075ed8dce781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:26.741269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f1546-3007-4af2-a237-075ed8dce781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:26.742655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f1546-3007-4af2-a237-075ed8dce781 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:26.754964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 342/701] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:26.760002 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0338a-dc2b-41d4-99f3-7fe52582c211 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:26.840813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:26.872596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0338a-dc2b-41d4-99f3-7fe52582c211 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:26.893463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922927c-52ad-41ba-820e-cbc5b8e0d74e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:26.894339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922927c-52ad-41ba-820e-cbc5b8e0d74e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:26.895134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922927c-52ad-41ba-820e-cbc5b8e0d74e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:26.898321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 343/702] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:26.905162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc6ccd-7eef-45bd-b8f2-210a4d8fc8ec None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:26.905729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc6ccd-7eef-45bd-b8f2-210a4d8fc8ec None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:26.906249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc6ccd-7eef-45bd-b8f2-210a4d8fc8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:26.910332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f52f48-f8f3-466c-88d7-ef4d3ab9932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:26.911225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f52f48-f8f3-466c-88d7-ef4d3ab9932e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:26.911503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f52f48-f8f3-466c-88d7-ef4d3ab9932e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:26.911778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f52f48-f8f3-466c-88d7-ef4d3ab9932e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:26.921078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f52f48-f8f3-466c-88d7-ef4d3ab9932e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:26.922570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f52f48-f8f3-466c-88d7-ef4d3ab9932e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:26.929661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3edc6ccd-7eef-45bd-b8f2-210a4d8fc8ec None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:26.931196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3edc6ccd-7eef-45bd-b8f2-210a4d8fc8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:26.933453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-07f52f48-f8f3-466c-88d7-ef4d3ab9932e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:26.940065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:26.951646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 360/703] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 13 08:48:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:26.952666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f52f48-f8f3-466c-88d7-ef4d3ab9932e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:27.021886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 344/704] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:48:27.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc6ccd-7eef-45bd-b8f2-210a4d8fc8ec None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:27.076039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f52f48-f8f3-466c-88d7-ef4d3ab9932e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:27.116144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:27.122432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6572893-1a2d-4cd2-a582-72286725bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:27.123167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6572893-1a2d-4cd2-a582-72286725bbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:27.123451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6572893-1a2d-4cd2-a582-72286725bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:27.123833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6572893-1a2d-4cd2-a582-72286725bbd8 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:27.124843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6572893-1a2d-4cd2-a582-72286725bbd8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:27.125154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6572893-1a2d-4cd2-a582-72286725bbd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:27.127515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6572893-1a2d-4cd2-a582-72286725bbd8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:27.150848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 361/705] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 13 08:48:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:27.158066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6572893-1a2d-4cd2-a582-72286725bbd8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:27.228397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc6ccd-7eef-45bd-b8f2-210a4d8fc8ec None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:27.315070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be871d-f71d-49c0-afef-542f804a807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:27.316827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be871d-f71d-49c0-afef-542f804a807e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:27.317374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be871d-f71d-49c0-afef-542f804a807e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:27.318716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be871d-f71d-49c0-afef-542f804a807e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:27.319579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be871d-f71d-49c0-afef-542f804a807e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:27.321871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be871d-f71d-49c0-afef-542f804a807e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:27.349310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6572893-1a2d-4cd2-a582-72286725bbd8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-05be871d-f71d-49c0-afef-542f804a807e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:27.349479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:27.350530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:27.351403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 345/706] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Jul 13 08:48:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:27.354841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be871d-f71d-49c0-afef-542f804a807e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:27.365436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:27.413749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27332977-a446-41d3-af8a-3c06b718fc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:27.414430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27332977-a446-41d3-af8a-3c06b718fc1e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:27.414711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27332977-a446-41d3-af8a-3c06b718fc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:27.414983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27332977-a446-41d3-af8a-3c06b718fc1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:27.515037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:27.534815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27332977-a446-41d3-af8a-3c06b718fc1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:27.536329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27332977-a446-41d3-af8a-3c06b718fc1e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:27.551575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be871d-f71d-49c0-afef-542f804a807e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:27.617080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e058f0-182e-489b-95a4-aa28bf2ae131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:27.617847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e058f0-182e-489b-95a4-aa28bf2ae131 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:27.618160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e058f0-182e-489b-95a4-aa28bf2ae131 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:27.618433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e058f0-182e-489b-95a4-aa28bf2ae131 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:27.619139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e058f0-182e-489b-95a4-aa28bf2ae131 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:27.619441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e058f0-182e-489b-95a4-aa28bf2ae131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:27.620800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e058f0-182e-489b-95a4-aa28bf2ae131 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:27.628721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 346/707] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 13 08:48:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:27.630771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306fb17-eaeb-4875-a1e7-9c29af75d576 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:27.631060 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306fb17-eaeb-4875-a1e7-9c29af75d576 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:27.631334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306fb17-eaeb-4875-a1e7-9c29af75d576 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:27.637762 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 347/708] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:27.639290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306fb17-eaeb-4875-a1e7-9c29af75d576 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:27.692329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 362/709] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 08:48:27.693955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d0e3e-adce-4617-9855-d05aa5c7d7ee None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:27.694490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d0e3e-adce-4617-9855-d05aa5c7d7ee None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:27.695098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d0e3e-adce-4617-9855-d05aa5c7d7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:27.712812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-161d0e3e-adce-4617-9855-d05aa5c7d7ee None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:27.714395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-161d0e3e-adce-4617-9855-d05aa5c7d7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:27.717686 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:27.758702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306fb17-eaeb-4875-a1e7-9c29af75d576 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:27.786875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 363/710] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:27.788331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d0e3e-adce-4617-9855-d05aa5c7d7ee None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:27.809119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:27.810859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb62da3-adbd-4bd0-8b36-d02d4de52798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:27.811686 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62da3-adbd-4bd0-8b36-d02d4de52798 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:27.812109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62da3-adbd-4bd0-8b36-d02d4de52798 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:27.812520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62da3-adbd-4bd0-8b36-d02d4de52798 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:27.813428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb62da3-adbd-4bd0-8b36-d02d4de52798 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:27.818268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb62da3-adbd-4bd0-8b36-d02d4de52798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:27.819941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb62da3-adbd-4bd0-8b36-d02d4de52798 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:27.831392 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 348/711] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 13 08:48:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:27.832475 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb62da3-adbd-4bd0-8b36-d02d4de52798 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:27.965075 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d0e3e-adce-4617-9855-d05aa5c7d7ee None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:28.038259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805d602-64c7-45f8-8cef-afbe5bec86ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:28.040404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805d602-64c7-45f8-8cef-afbe5bec86ef None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.040685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805d602-64c7-45f8-8cef-afbe5bec86ef None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.040952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805d602-64c7-45f8-8cef-afbe5bec86ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.054393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb62da3-adbd-4bd0-8b36-d02d4de52798 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:28.055713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805d602-64c7-45f8-8cef-afbe5bec86ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:28.057395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805d602-64c7-45f8-8cef-afbe5bec86ef None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-9805d602-64c7-45f8-8cef-afbe5bec86ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:28.079172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:28.080741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:28.081627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:28.081627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:28.081627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:28.081627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:28.084043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 364/712] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:28.087837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:28.089870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805d602-64c7-45f8-8cef-afbe5bec86ef None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:28.091559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d6246-22c1-4ef9-8b71-02519363f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:28.092354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d6246-22c1-4ef9-8b71-02519363f586 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.101588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d6246-22c1-4ef9-8b71-02519363f586 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.101588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d6246-22c1-4ef9-8b71-02519363f586 None admin] PATH_INFO: `/v3/services/network` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.101588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d6246-22c1-4ef9-8b71-02519363f586 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:28.101588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d6246-22c1-4ef9-8b71-02519363f586 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-043d6246-22c1-4ef9-8b71-02519363f586 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:28.108331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:28.109344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 08:48:28.110386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:28.112904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 349/713] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:27 2022] GET /identity/v3/services/network => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:28.117195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d6246-22c1-4ef9-8b71-02519363f586 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:28.177482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:28.211499 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805d602-64c7-45f8-8cef-afbe5bec86ef None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:28.236299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d6246-22c1-4ef9-8b71-02519363f586 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:28.247834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047f1d0-e2c3-4513-ba2d-3e63f3cde015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:28.248772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047f1d0-e2c3-4513-ba2d-3e63f3cde015 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.249296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047f1d0-e2c3-4513-ba2d-3e63f3cde015 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.249925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047f1d0-e2c3-4513-ba2d-3e63f3cde015 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.250873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047f1d0-e2c3-4513-ba2d-3e63f3cde015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:28.252119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047f1d0-e2c3-4513-ba2d-3e63f3cde015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:28.253712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047f1d0-e2c3-4513-ba2d-3e63f3cde015 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:28.270233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 365/714] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:28.272422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047f1d0-e2c3-4513-ba2d-3e63f3cde015 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:28.274274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e00f5-d50d-4cad-873a-1a25f6e26b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:28.274935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e00f5-d50d-4cad-873a-1a25f6e26b84 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.275212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e00f5-d50d-4cad-873a-1a25f6e26b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.275486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e00f5-d50d-4cad-873a-1a25f6e26b84 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.276185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e00f5-d50d-4cad-873a-1a25f6e26b84 None admin] RBAC: Adding query filter params (name=network) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:28.276486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e00f5-d50d-4cad-873a-1a25f6e26b84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:28.278527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e00f5-d50d-4cad-873a-1a25f6e26b84 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:28.287132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 350/715] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:48:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:28.288533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e00f5-d50d-4cad-873a-1a25f6e26b84 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:28.364764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:28.395272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047f1d0-e2c3-4513-ba2d-3e63f3cde015 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:28.416313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e00f5-d50d-4cad-873a-1a25f6e26b84 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:28.426461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:28.431579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d9920-219d-4ff9-b30b-4f0dfc06c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:28.432567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d9920-219d-4ff9-b30b-4f0dfc06c99c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.432854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d9920-219d-4ff9-b30b-4f0dfc06c99c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.433213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d9920-219d-4ff9-b30b-4f0dfc06c99c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.441156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d9920-219d-4ff9-b30b-4f0dfc06c99c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:28.442922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d9920-219d-4ff9-b30b-4f0dfc06c99c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f17d9920-219d-4ff9-b30b-4f0dfc06c99c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:28.457946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:28.459114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:28.460220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:28.460220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:28.460220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:28.460220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:28.460220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:28.460220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:28.460220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:28.460220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:28.462263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 366/716] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:28.462504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcfef0-bad3-449a-bdc3-1b5dce4967aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:28.466760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b23287-07d9-4405-a05b-de264968f4a0 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.466760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b23287-07d9-4405-a05b-de264968f4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.466760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b23287-07d9-4405-a05b-de264968f4a0 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.466760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 367/717] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:28.467242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcfef0-bad3-449a-bdc3-1b5dce4967aa None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.467608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcfef0-bad3-449a-bdc3-1b5dce4967aa None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.467975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcfef0-bad3-449a-bdc3-1b5dce4967aa None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.468774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcfef0-bad3-449a-bdc3-1b5dce4967aa None admin] RBAC: Adding query filter params (type=network) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:28.469569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcfef0-bad3-449a-bdc3-1b5dce4967aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:28.471166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f26632-0eba-4338-b7aa-328906ee992f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.471555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f26632-0eba-4338-b7aa-328906ee992f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.471918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f26632-0eba-4338-b7aa-328906ee992f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.474626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcfef0-bad3-449a-bdc3-1b5dce4967aa None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:28.486478 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 351/718] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:48:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:28.487529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcfef0-bad3-449a-bdc3-1b5dce4967aa None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:28.495211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-58f26632-0eba-4338-b7aa-328906ee992f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:28.496668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-58f26632-0eba-4338-b7aa-328906ee992f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:28.640776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 368/719] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:28.650658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcfef0-bad3-449a-bdc3-1b5dce4967aa None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:28.680839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:28.689573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f26632-0eba-4338-b7aa-328906ee992f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:28.703662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c40e57-328f-47b4-89da-7059be51a850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:28.704499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c40e57-328f-47b4-89da-7059be51a850 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.705431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c40e57-328f-47b4-89da-7059be51a850 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.705711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c40e57-328f-47b4-89da-7059be51a850 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.706773 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c40e57-328f-47b4-89da-7059be51a850 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:28.707083 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c40e57-328f-47b4-89da-7059be51a850 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:28.708537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c40e57-328f-47b4-89da-7059be51a850 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:28.726552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 352/720] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:28.728617 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c40e57-328f-47b4-89da-7059be51a850 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:28.808499 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f26632-0eba-4338-b7aa-328906ee992f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:28.831296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:28.842499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c40e57-328f-47b4-89da-7059be51a850 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:28.847988 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3a509-cf37-4405-9874-d20005599b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:28.849928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3a509-cf37-4405-9874-d20005599b47 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.850450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3a509-cf37-4405-9874-d20005599b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.850939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3a509-cf37-4405-9874-d20005599b47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.862224 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3a509-cf37-4405-9874-d20005599b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3ec3a509-cf37-4405-9874-d20005599b47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:28.871494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:28.872452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:28.873290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:28.876145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 369/721] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:28.881097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51519b-c5d8-4a35-8194-69a2fbce834c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:28.881884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3a509-cf37-4405-9874-d20005599b47 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:28.882327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51519b-c5d8-4a35-8194-69a2fbce834c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:28.882819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51519b-c5d8-4a35-8194-69a2fbce834c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:28.883280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51519b-c5d8-4a35-8194-69a2fbce834c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:28.884291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51519b-c5d8-4a35-8194-69a2fbce834c None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=8af26e036f4e466ebb923771c1f5f502) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:28.884794 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51519b-c5d8-4a35-8194-69a2fbce834c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:28.886453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51519b-c5d8-4a35-8194-69a2fbce834c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:28.895351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 353/722] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:28 2022] GET /identity/v3/role_assignments?user.id=8af26e036f4e466ebb923771c1f5f502&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:29.064995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3a509-cf37-4405-9874-d20005599b47 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:29.108185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef3671-3fee-4a6e-ba26-53f4c72af2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:29.109265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef3671-3fee-4a6e-ba26-53f4c72af2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:29.109750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef3671-3fee-4a6e-ba26-53f4c72af2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:29.110319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef3671-3fee-4a6e-ba26-53f4c72af2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:29.111876 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef3671-3fee-4a6e-ba26-53f4c72af2a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:29.112437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef3671-3fee-4a6e-ba26-53f4c72af2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:29.114008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef3671-3fee-4a6e-ba26-53f4c72af2a0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:29.128925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 370/723] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:29.134588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51519b-c5d8-4a35-8194-69a2fbce834c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:29.180855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d497d-b4e5-4f87-a201-316fe4af7c14 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:29.181181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d497d-b4e5-4f87-a201-316fe4af7c14 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:29.181409 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d497d-b4e5-4f87-a201-316fe4af7c14 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:29.182905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 371/724] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:29.193284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b8e6e-fe63-473a-a51a-6843e6b93eec None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:29.193565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b8e6e-fe63-473a-a51a-6843e6b93eec None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:29.193829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b8e6e-fe63-473a-a51a-6843e6b93eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:29.210595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:29.217544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a92b8e6e-fe63-473a-a51a-6843e6b93eec None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:29.219661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a92b8e6e-fe63-473a-a51a-6843e6b93eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:29.275101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:29.316128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 372/725] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 08:48:29.336518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51519b-c5d8-4a35-8194-69a2fbce834c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:29.350296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b8e6e-fe63-473a-a51a-6843e6b93eec None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:29.440593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec041-9120-4bd4-94b2-9ffbccf29149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:29.441395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec041-9120-4bd4-94b2-9ffbccf29149 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:29.441677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec041-9120-4bd4-94b2-9ffbccf29149 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:29.441954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ec041-9120-4bd4-94b2-9ffbccf29149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:29.460713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b8e6e-fe63-473a-a51a-6843e6b93eec None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:29.470602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ec041-9120-4bd4-94b2-9ffbccf29149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:29.472455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ec041-9120-4bd4-94b2-9ffbccf29149 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:29.500134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc320ebd-2ccd-43bd-aa30-aa1b86ef2230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:29.500919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc320ebd-2ccd-43bd-aa30-aa1b86ef2230 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:29.501697 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc320ebd-2ccd-43bd-aa30-aa1b86ef2230 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:29.501978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc320ebd-2ccd-43bd-aa30-aa1b86ef2230 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ed0ec041-9120-4bd4-94b2-9ffbccf29149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:29.514691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:29.515587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:29.516505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:29.516505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:29.516505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:29.516505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:29.516737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc320ebd-2ccd-43bd-aa30-aa1b86ef2230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:29.519410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 354/726] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-fc320ebd-2ccd-43bd-aa30-aa1b86ef2230 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:29.524030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:29.524999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:29.525863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:29.527519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec041-9120-4bd4-94b2-9ffbccf29149 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:29.529948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 373/727] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:29.531202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc320ebd-2ccd-43bd-aa30-aa1b86ef2230 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:29.568772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:29.659186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc320ebd-2ccd-43bd-aa30-aa1b86ef2230 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:29.730383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:29.730794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02addc4-2a5b-4c9b-9293-fc033a5ce733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:29.731653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02addc4-2a5b-4c9b-9293-fc033a5ce733 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:29.731782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02addc4-2a5b-4c9b-9293-fc033a5ce733 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:29.731992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02addc4-2a5b-4c9b-9293-fc033a5ce733 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:29.732734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02addc4-2a5b-4c9b-9293-fc033a5ce733 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:29.733031 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02addc4-2a5b-4c9b-9293-fc033a5ce733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:29.734415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02addc4-2a5b-4c9b-9293-fc033a5ce733 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:29.738688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ec041-9120-4bd4-94b2-9ffbccf29149 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:29.762209 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 374/728] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:29.768899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02addc4-2a5b-4c9b-9293-fc033a5ce733 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:29.793106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac5f1a-1262-4b19-b692-82ff6dbdda10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:29.793783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac5f1a-1262-4b19-b692-82ff6dbdda10 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:29.794080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac5f1a-1262-4b19-b692-82ff6dbdda10 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:29.796217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac5f1a-1262-4b19-b692-82ff6dbdda10 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:29.796943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac5f1a-1262-4b19-b692-82ff6dbdda10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:29.797248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac5f1a-1262-4b19-b692-82ff6dbdda10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:29.800043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac5f1a-1262-4b19-b692-82ff6dbdda10 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:29.827966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 355/729] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:29.831282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac5f1a-1262-4b19-b692-82ff6dbdda10 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:29.925144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:29.933352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02addc4-2a5b-4c9b-9293-fc033a5ce733 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:29.980471 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b4357-69cd-4782-8a22-1255a5064286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:29.981270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b4357-69cd-4782-8a22-1255a5064286 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:29.981551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b4357-69cd-4782-8a22-1255a5064286 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:29.981824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b4357-69cd-4782-8a22-1255a5064286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:30.008320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b4357-69cd-4782-8a22-1255a5064286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:30.009781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b4357-69cd-4782-8a22-1255a5064286 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c68b4357-69cd-4782-8a22-1255a5064286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:30.033815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:30.034898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:30.036271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:30.036271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:30.036271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:30.036271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:30.036271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 375/730] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:30.043101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b4357-69cd-4782-8a22-1255a5064286 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:30.064364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac5f1a-1262-4b19-b692-82ff6dbdda10 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:30.123652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52f689-df9a-4ce7-9989-904274d632f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:30.124475 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52f689-df9a-4ce7-9989-904274d632f6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:30.124759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52f689-df9a-4ce7-9989-904274d632f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:30.125033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52f689-df9a-4ce7-9989-904274d632f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:30.140284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52f689-df9a-4ce7-9989-904274d632f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:30.141721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52f689-df9a-4ce7-9989-904274d632f6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-bb52f689-df9a-4ce7-9989-904274d632f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:30.183445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:30.184461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:30.186369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 356/731] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 13 08:48:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:30.190734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52f689-df9a-4ce7-9989-904274d632f6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:30.198161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b4357-69cd-4782-8a22-1255a5064286 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:30.234691 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886ab12-782a-443b-8a1b-cd96bc76ecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:30.235366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886ab12-782a-443b-8a1b-cd96bc76ecef None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:30.235495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:30.236240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886ab12-782a-443b-8a1b-cd96bc76ecef None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:30.236549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886ab12-782a-443b-8a1b-cd96bc76ecef None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:30.237385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886ab12-782a-443b-8a1b-cd96bc76ecef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:30.237692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886ab12-782a-443b-8a1b-cd96bc76ecef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:30.239117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886ab12-782a-443b-8a1b-cd96bc76ecef None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:30.252992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 376/732] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:30.258478 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886ab12-782a-443b-8a1b-cd96bc76ecef None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:30.268643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:30.363125 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886ab12-782a-443b-8a1b-cd96bc76ecef None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:30.399209 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9333542-6de8-4056-a839-5b20410151b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:30.399930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9333542-6de8-4056-a839-5b20410151b4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:30.400214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9333542-6de8-4056-a839-5b20410151b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:30.400490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9333542-6de8-4056-a839-5b20410151b4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:30.408192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9333542-6de8-4056-a839-5b20410151b4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:30.409643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9333542-6de8-4056-a839-5b20410151b4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e9333542-6de8-4056-a839-5b20410151b4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:30.424582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:30.425512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:30.427566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 377/733] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:30.428161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52f689-df9a-4ce7-9989-904274d632f6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:30.428247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc21025-bd7a-49c1-b6ef-c8e7aba62ff4 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:30.428247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc21025-bd7a-49c1-b6ef-c8e7aba62ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:30.428247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc21025-bd7a-49c1-b6ef-c8e7aba62ff4 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:30.433657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 378/734] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:30.434483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc21025-bd7a-49c1-b6ef-c8e7aba62ff4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:30.468244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7b178-5b84-4e3d-8156-85d65c17002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:30.469004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7b178-5b84-4e3d-8156-85d65c17002a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:30.469299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7b178-5b84-4e3d-8156-85d65c17002a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:30.469668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7b178-5b84-4e3d-8156-85d65c17002a None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:30.470670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7b178-5b84-4e3d-8156-85d65c17002a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:30.470983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7b178-5b84-4e3d-8156-85d65c17002a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:30.472635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7b178-5b84-4e3d-8156-85d65c17002a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:30.492185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 357/735] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 13 08:48:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:30.493623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8b9b5-1668-4d0c-bd28-1808bc922d61 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:30.494007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8b9b5-1668-4d0c-bd28-1808bc922d61 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:30.494319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8b9b5-1668-4d0c-bd28-1808bc922d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:30.503993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:30.513183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-66b8b9b5-1668-4d0c-bd28-1808bc922d61 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:30.514381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-66b8b9b5-1668-4d0c-bd28-1808bc922d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:30.541255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc21025-bd7a-49c1-b6ef-c8e7aba62ff4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:30.578718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee934f1-5800-45d4-8fd4-039f9894a3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:30.579603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee934f1-5800-45d4-8fd4-039f9894a3fc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:30.579954 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee934f1-5800-45d4-8fd4-039f9894a3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:30.580364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee934f1-5800-45d4-8fd4-039f9894a3fc None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:30.581332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee934f1-5800-45d4-8fd4-039f9894a3fc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:30.581710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee934f1-5800-45d4-8fd4-039f9894a3fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:30.583316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee934f1-5800-45d4-8fd4-039f9894a3fc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:30.594369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 358/736] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:30.595388 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8b9b5-1668-4d0c-bd28-1808bc922d61 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:30.604621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 379/737] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 13 08:48:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:30.607999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:30.610011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee934f1-5800-45d4-8fd4-039f9894a3fc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:30.714700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee934f1-5800-45d4-8fd4-039f9894a3fc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:30.724502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8b9b5-1668-4d0c-bd28-1808bc922d61 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:30.739939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:30.750666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282fa09-6015-4ba2-a998-2ee2f81c40b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:30.756938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282fa09-6015-4ba2-a998-2ee2f81c40b4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:30.756938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282fa09-6015-4ba2-a998-2ee2f81c40b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:30.756938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282fa09-6015-4ba2-a998-2ee2f81c40b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:30.764217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08122580-7391-4ddb-a21f-16f5a4d19fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:30.765055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08122580-7391-4ddb-a21f-16f5a4d19fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:30.765506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08122580-7391-4ddb-a21f-16f5a4d19fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:30.765945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08122580-7391-4ddb-a21f-16f5a4d19fc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:30.825592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282fa09-6015-4ba2-a998-2ee2f81c40b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:30.827082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282fa09-6015-4ba2-a998-2ee2f81c40b4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:30.841339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08122580-7391-4ddb-a21f-16f5a4d19fc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:30.843260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08122580-7391-4ddb-a21f-16f5a4d19fc2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:30.886514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:30.938133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 380/738] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:30.939030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282fa09-6015-4ba2-a998-2ee2f81c40b4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:30.955837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 359/739] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 08:48:30.957009 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08122580-7391-4ddb-a21f-16f5a4d19fc2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:30.964292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:31.042489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282fa09-6015-4ba2-a998-2ee2f81c40b4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:31.071407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08122580-7391-4ddb-a21f-16f5a4d19fc2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:31.078230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9ff0e-c702-478f-8c28-5eb519ed6d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:31.078861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9ff0e-c702-478f-8c28-5eb519ed6d5b None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.079464 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9ff0e-c702-478f-8c28-5eb519ed6d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.079576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9ff0e-c702-478f-8c28-5eb519ed6d5b None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.080563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9ff0e-c702-478f-8c28-5eb519ed6d5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:31.081928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:31.082133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9ff0e-c702-478f-8c28-5eb519ed6d5b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:31.106695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3fd26-cc44-46c6-837a-3c83017ee9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:31.107478 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3fd26-cc44-46c6-837a-3c83017ee9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.107780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3fd26-cc44-46c6-837a-3c83017ee9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.108054 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3fd26-cc44-46c6-837a-3c83017ee9f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.119423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3fd26-cc44-46c6-837a-3c83017ee9f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:31.120920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3fd26-cc44-46c6-837a-3c83017ee9f6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7ba3fd26-cc44-46c6-837a-3c83017ee9f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:31.139354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:31.140276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:31.141247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:31.141247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:31.141247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:31.141247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:31.142379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 360/740] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:31.143091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3fd26-cc44-46c6-837a-3c83017ee9f6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:31.178676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 381/741] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:48:30 2022] POST /identity/v3/services => generated 251 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:31.191625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa875e-9d43-4a54-a71c-16777a548a63 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.191962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa875e-9d43-4a54-a71c-16777a548a63 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.192251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa875e-9d43-4a54-a71c-16777a548a63 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.193681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 382/742] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:31.194321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa875e-9d43-4a54-a71c-16777a548a63 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:31.266733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3fd26-cc44-46c6-837a-3c83017ee9f6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:31.308900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:31.320011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad225de-22a3-4638-9918-8b8885e29c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:31.323133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad225de-22a3-4638-9918-8b8885e29c82 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.323133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad225de-22a3-4638-9918-8b8885e29c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.323133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad225de-22a3-4638-9918-8b8885e29c82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.328040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa875e-9d43-4a54-a71c-16777a548a63 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:31.334394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad225de-22a3-4638-9918-8b8885e29c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:31.336485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad225de-22a3-4638-9918-8b8885e29c82 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-4ad225de-22a3-4638-9918-8b8885e29c82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:31.361145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:31.362113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:31.363048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:31.363048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:31.363048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:31.363048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:31.363048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 361/743] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:31.364074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52f7c8-d3ea-48ba-9659-9cadc6100ad0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.364350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52f7c8-d3ea-48ba-9659-9cadc6100ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.364622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52f7c8-d3ea-48ba-9659-9cadc6100ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.374073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6d799-b21e-4355-a987-6039c4dd4046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:31.374851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6d799-b21e-4355-a987-6039c4dd4046 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.375212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6d799-b21e-4355-a987-6039c4dd4046 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.375724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6d799-b21e-4355-a987-6039c4dd4046 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.376637 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6d799-b21e-4355-a987-6039c4dd4046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:31.377009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6d799-b21e-4355-a987-6039c4dd4046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:31.377875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:31.378582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6d799-b21e-4355-a987-6039c4dd4046 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:31.384233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3b52f7c8-d3ea-48ba-9659-9cadc6100ad0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:31.385610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3b52f7c8-d3ea-48ba-9659-9cadc6100ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:31.392723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 383/744] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:31.394025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bea29-7e37-4ae6-a8fc-31ee3da82e02 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.394439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bea29-7e37-4ae6-a8fc-31ee3da82e02 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.394726 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bea29-7e37-4ae6-a8fc-31ee3da82e02 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.397137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 384/745] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:31.397137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bea29-7e37-4ae6-a8fc-31ee3da82e02 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:31.460578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 362/746] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:31.461237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52f7c8-d3ea-48ba-9659-9cadc6100ad0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:31.520964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bea29-7e37-4ae6-a8fc-31ee3da82e02 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:31.556555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818533f-2aaa-4249-a6ad-ab7ba54a8bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:31.557327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818533f-2aaa-4249-a6ad-ab7ba54a8bbd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.557607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818533f-2aaa-4249-a6ad-ab7ba54a8bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.557996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818533f-2aaa-4249-a6ad-ab7ba54a8bbd None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.558740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818533f-2aaa-4249-a6ad-ab7ba54a8bbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:31.559096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818533f-2aaa-4249-a6ad-ab7ba54a8bbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:31.560526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818533f-2aaa-4249-a6ad-ab7ba54a8bbd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:31.564344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52f7c8-d3ea-48ba-9659-9cadc6100ad0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:31.578593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 385/747] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:31.579720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ed662-344f-4bed-a685-ba0e473d8d91 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.580036 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ed662-344f-4bed-a685-ba0e473d8d91 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.580290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ed662-344f-4bed-a685-ba0e473d8d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.586425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:31.596888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e68ed662-344f-4bed-a685-ba0e473d8d91 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:31.598488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e68ed662-344f-4bed-a685-ba0e473d8d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:31.600572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992effa-0cd5-410c-b8d5-cbab3c290d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:31.601359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992effa-0cd5-410c-b8d5-cbab3c290d3e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.601640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992effa-0cd5-410c-b8d5-cbab3c290d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.601910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992effa-0cd5-410c-b8d5-cbab3c290d3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.605042 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:31.609754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992effa-0cd5-410c-b8d5-cbab3c290d3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:31.611755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992effa-0cd5-410c-b8d5-cbab3c290d3e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-4992effa-0cd5-410c-b8d5-cbab3c290d3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:31.625654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:31.627389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:31.628295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:31.628295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:31.628295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:31.628295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:31.628295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:31.628295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:31.628295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:31.628295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:31.632544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 363/748] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:31.633367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992effa-0cd5-410c-b8d5-cbab3c290d3e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:31.682068 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 386/749] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 08:48:31.682068 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ed662-344f-4bed-a685-ba0e473d8d91 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:31.757240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992effa-0cd5-410c-b8d5-cbab3c290d3e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:31.785840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ed662-344f-4bed-a685-ba0e473d8d91 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:31.792819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de040ff2-e294-40ab-95dd-eb5f8bd36192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:31.793900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de040ff2-e294-40ab-95dd-eb5f8bd36192 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.794267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de040ff2-e294-40ab-95dd-eb5f8bd36192 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.794545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de040ff2-e294-40ab-95dd-eb5f8bd36192 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.795898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de040ff2-e294-40ab-95dd-eb5f8bd36192 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:31.797681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de040ff2-e294-40ab-95dd-eb5f8bd36192 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:31.821168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bbc4a-1e53-45d2-a464-64f89ca03903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:31.821884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bbc4a-1e53-45d2-a464-64f89ca03903 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:31.822186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bbc4a-1e53-45d2-a464-64f89ca03903 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:31.822458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bbc4a-1e53-45d2-a464-64f89ca03903 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:31.824456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:31.830225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bbc4a-1e53-45d2-a464-64f89ca03903 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:31.831647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bbc4a-1e53-45d2-a464-64f89ca03903 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8e0bbc4a-1e53-45d2-a464-64f89ca03903 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:31.845734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:31.846801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:31.847730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:31.847730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:31.847730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:31.847730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:31.847730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:31.847730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:31.847730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:31.847730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:31.847730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 387/750] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:31.847730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bbc4a-1e53-45d2-a464-64f89ca03903 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:31.868597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 364/751] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:48:31 2022] POST /identity/v3/services => generated 240 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:31.869725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de040ff2-e294-40ab-95dd-eb5f8bd36192 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:31.969606 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:31.971701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bbc4a-1e53-45d2-a464-64f89ca03903 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:32.026669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914943f-9cfc-4164-955b-a418f52b55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:32.027337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914943f-9cfc-4164-955b-a418f52b55d8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:32.027620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914943f-9cfc-4164-955b-a418f52b55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:32.027893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914943f-9cfc-4164-955b-a418f52b55d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:32.028681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914943f-9cfc-4164-955b-a418f52b55d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:32.028993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914943f-9cfc-4164-955b-a418f52b55d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:32.030521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914943f-9cfc-4164-955b-a418f52b55d8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:32.034547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de040ff2-e294-40ab-95dd-eb5f8bd36192 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:32.049599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 388/752] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:32.050959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914943f-9cfc-4164-955b-a418f52b55d8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:32.085981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5d7ba-5875-4956-9e5e-764748f36fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:32.086760 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5d7ba-5875-4956-9e5e-764748f36fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:32.087112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5d7ba-5875-4956-9e5e-764748f36fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:32.087355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5d7ba-5875-4956-9e5e-764748f36fe5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:32.099061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5d7ba-5875-4956-9e5e-764748f36fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-10e5d7ba-5875-4956-9e5e-764748f36fe5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:32.110081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:32.111856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:32.112985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 365/753] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:32.113850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5d7ba-5875-4956-9e5e-764748f36fe5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:32.118449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:32.170730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914943f-9cfc-4164-955b-a418f52b55d8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:32.208943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b28939-7d37-4dee-ba0c-2a4819e5e10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:32.209663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b28939-7d37-4dee-ba0c-2a4819e5e10e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:32.209951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b28939-7d37-4dee-ba0c-2a4819e5e10e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:32.210298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b28939-7d37-4dee-ba0c-2a4819e5e10e None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:32.211196 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b28939-7d37-4dee-ba0c-2a4819e5e10e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:32.211513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b28939-7d37-4dee-ba0c-2a4819e5e10e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:32.213094 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b28939-7d37-4dee-ba0c-2a4819e5e10e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:32.226302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 389/754] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:32.227206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b28939-7d37-4dee-ba0c-2a4819e5e10e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:32.227666 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5d7ba-5875-4956-9e5e-764748f36fe5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:32.245423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:32.264727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9a408-1510-4da5-ac76-b3c049d5f2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:32.265529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9a408-1510-4da5-ac76-b3c049d5f2da None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:32.265917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9a408-1510-4da5-ac76-b3c049d5f2da None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:32.266315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9a408-1510-4da5-ac76-b3c049d5f2da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:32.267290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9a408-1510-4da5-ac76-b3c049d5f2da None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=906fc971a5ee4ffd9503ab13d46cd605) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:32.267812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9a408-1510-4da5-ac76-b3c049d5f2da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:32.269353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9a408-1510-4da5-ac76-b3c049d5f2da None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:32.275736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 366/755] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:32 2022] GET /identity/v3/role_assignments?user.id=906fc971a5ee4ffd9503ab13d46cd605&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:32.276717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9a408-1510-4da5-ac76-b3c049d5f2da None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:32.342953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b28939-7d37-4dee-ba0c-2a4819e5e10e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:32.373854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:32.386856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf816d1-6939-4c0d-a56f-7a68ea70854e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:32.391012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf816d1-6939-4c0d-a56f-7a68ea70854e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:32.391012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf816d1-6939-4c0d-a56f-7a68ea70854e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:32.391012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf816d1-6939-4c0d-a56f-7a68ea70854e None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:32.391012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf816d1-6939-4c0d-a56f-7a68ea70854e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:32.391012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf816d1-6939-4c0d-a56f-7a68ea70854e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:32.391617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf816d1-6939-4c0d-a56f-7a68ea70854e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:32.424396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 390/756] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:32.426347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf816d1-6939-4c0d-a56f-7a68ea70854e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:32.475471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9a408-1510-4da5-ac76-b3c049d5f2da None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:32.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17689fd6-1add-44b5-9ead-2f866ce9dcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:32.530090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17689fd6-1add-44b5-9ead-2f866ce9dcd3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:32.530375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17689fd6-1add-44b5-9ead-2f866ce9dcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:32.530653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17689fd6-1add-44b5-9ead-2f866ce9dcd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:32.531482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17689fd6-1add-44b5-9ead-2f866ce9dcd3 None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=0b380bc85da64a8bb687e2e5c4577e79) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:32.531793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17689fd6-1add-44b5-9ead-2f866ce9dcd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:32.533264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17689fd6-1add-44b5-9ead-2f866ce9dcd3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:32.548440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 367/757] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:32 2022] GET /identity/v3/role_assignments?user.id=0b380bc85da64a8bb687e2e5c4577e79&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:32.606644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf816d1-6939-4c0d-a56f-7a68ea70854e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:32.660413 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e533bb6-c37a-4127-bbd8-d6f9fc1f68bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:32.661226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e533bb6-c37a-4127-bbd8-d6f9fc1f68bb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:32.661502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e533bb6-c37a-4127-bbd8-d6f9fc1f68bb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:32.661780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e533bb6-c37a-4127-bbd8-d6f9fc1f68bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:32.678618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e533bb6-c37a-4127-bbd8-d6f9fc1f68bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:32.680112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e533bb6-c37a-4127-bbd8-d6f9fc1f68bb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1e533bb6-c37a-4127-bbd8-d6f9fc1f68bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:32.701302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:32.702329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:32.703428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:32.703428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:32.703428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:32.703428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:32.703428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 391/758] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:32.708371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17689fd6-1add-44b5-9ead-2f866ce9dcd3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:32.778419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:32.839985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17689fd6-1add-44b5-9ead-2f866ce9dcd3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:32.878314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52b3e0-349b-4ac2-b1ce-eb5982382a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:32.878901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52b3e0-349b-4ac2-b1ce-eb5982382a4a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:32.879185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52b3e0-349b-4ac2-b1ce-eb5982382a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:32.879464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52b3e0-349b-4ac2-b1ce-eb5982382a4a None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:32.880185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52b3e0-349b-4ac2-b1ce-eb5982382a4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:32.880550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52b3e0-349b-4ac2-b1ce-eb5982382a4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:32.881917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52b3e0-349b-4ac2-b1ce-eb5982382a4a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:32.895076 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 368/759] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:32.900365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e533bb6-c37a-4127-bbd8-d6f9fc1f68bb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:32.953598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:33.043608 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e533bb6-c37a-4127-bbd8-d6f9fc1f68bb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:33.083644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674856a-74d5-4123-b1bd-4db243820516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:33.084437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674856a-74d5-4123-b1bd-4db243820516 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:33.084718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674856a-74d5-4123-b1bd-4db243820516 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:33.084991 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674856a-74d5-4123-b1bd-4db243820516 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:33.092894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674856a-74d5-4123-b1bd-4db243820516 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:33.094368 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674856a-74d5-4123-b1bd-4db243820516 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b674856a-74d5-4123-b1bd-4db243820516 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:33.108641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:33.109627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:33.111122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 392/760] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 13 08:48:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:33.116405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52b3e0-349b-4ac2-b1ce-eb5982382a4a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:33.264257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52b3e0-349b-4ac2-b1ce-eb5982382a4a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:33.271014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:33.316048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80559d-4076-4684-8111-e7c4b9079f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:33.317113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80559d-4076-4684-8111-e7c4b9079f7e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:33.317645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80559d-4076-4684-8111-e7c4b9079f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:33.318150 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80559d-4076-4684-8111-e7c4b9079f7e None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:33.319197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80559d-4076-4684-8111-e7c4b9079f7e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:33.319709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80559d-4076-4684-8111-e7c4b9079f7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:33.321351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80559d-4076-4684-8111-e7c4b9079f7e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:33.352259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 369/761] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 13 08:48:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:33.357852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674856a-74d5-4123-b1bd-4db243820516 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:33.478804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:33.498604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674856a-74d5-4123-b1bd-4db243820516 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:33.550338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfcee2-ab48-4b6a-9c7b-7715e8323dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:33.550539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8242d2-61fd-4b92-89f8-c74922683643 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:33.550771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8242d2-61fd-4b92-89f8-c74922683643 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:33.551040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8242d2-61fd-4b92-89f8-c74922683643 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:33.551944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfcee2-ab48-4b6a-9c7b-7715e8323dcd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:33.552425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfcee2-ab48-4b6a-9c7b-7715e8323dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:33.552665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 370/762] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:33.553123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfcee2-ab48-4b6a-9c7b-7715e8323dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:33.566196 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfcee2-ab48-4b6a-9c7b-7715e8323dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:33.567032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428ceae-d7c0-45eb-82d2-845abc87b102 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:33.567323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428ceae-d7c0-45eb-82d2-845abc87b102 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:33.567605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428ceae-d7c0-45eb-82d2-845abc87b102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:33.568576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfcee2-ab48-4b6a-9c7b-7715e8323dcd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-02cfcee2-ab48-4b6a-9c7b-7715e8323dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:33.587463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:33.588377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:33.589254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:33.589254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:33.589254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:33.589254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:33.591603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 393/763] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:33.600170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfcee2-ab48-4b6a-9c7b-7715e8323dcd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:33.605053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f428ceae-d7c0-45eb-82d2-845abc87b102 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:33.606259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f428ceae-d7c0-45eb-82d2-845abc87b102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:33.722381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfcee2-ab48-4b6a-9c7b-7715e8323dcd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:33.743979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 371/764] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:33.760068 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f2157-90f4-4f8c-b74e-3e9799e3f855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:33.760741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f2157-90f4-4f8c-b74e-3e9799e3f855 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:33.761020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f2157-90f4-4f8c-b74e-3e9799e3f855 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:33.761293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f2157-90f4-4f8c-b74e-3e9799e3f855 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:33.762013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f2157-90f4-4f8c-b74e-3e9799e3f855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:33.762341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f2157-90f4-4f8c-b74e-3e9799e3f855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:33.763728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f2157-90f4-4f8c-b74e-3e9799e3f855 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:33.766140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:33.776734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 394/765] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:33.782547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428ceae-d7c0-45eb-82d2-845abc87b102 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:33.791145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f2157-90f4-4f8c-b74e-3e9799e3f855 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:33.901307 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f2157-90f4-4f8c-b74e-3e9799e3f855 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:33.939871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344bf03-b0de-45e4-9c05-d97074123bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:33.940741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344bf03-b0de-45e4-9c05-d97074123bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:33.941031 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344bf03-b0de-45e4-9c05-d97074123bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:33.941312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344bf03-b0de-45e4-9c05-d97074123bf1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:33.941975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344bf03-b0de-45e4-9c05-d97074123bf1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:33.943943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344bf03-b0de-45e4-9c05-d97074123bf1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c344bf03-b0de-45e4-9c05-d97074123bf1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:33.959527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:33.960463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:33.961440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 395/766] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Jul 13 08:48:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:33.969228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344bf03-b0de-45e4-9c05-d97074123bf1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:34.021261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:34.037095 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:34.038652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428ceae-d7c0-45eb-82d2-845abc87b102 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:34.090494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344bf03-b0de-45e4-9c05-d97074123bf1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:34.105722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b520f-a64b-4f0f-a15e-8cae2455cd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:34.106569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b520f-a64b-4f0f-a15e-8cae2455cd5c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.106852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b520f-a64b-4f0f-a15e-8cae2455cd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.107173 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b520f-a64b-4f0f-a15e-8cae2455cd5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.123318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b520f-a64b-4f0f-a15e-8cae2455cd5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:34.124788 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b520f-a64b-4f0f-a15e-8cae2455cd5c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:34.136909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e100d-4557-4804-b2b4-c79d49b365a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:34.137855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e100d-4557-4804-b2b4-c79d49b365a9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.138386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e100d-4557-4804-b2b4-c79d49b365a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.138862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e100d-4557-4804-b2b4-c79d49b365a9 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.139764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e100d-4557-4804-b2b4-c79d49b365a9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:34.140288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e100d-4557-4804-b2b4-c79d49b365a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-758b520f-a64b-4f0f-a15e-8cae2455cd5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:34.141007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:34.142626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:34.143547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:34.143547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:34.143547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:34.143547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:34.143547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:34.143547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:34.143547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:34.143547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:34.143547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 372/767] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:34.149443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b520f-a64b-4f0f-a15e-8cae2455cd5c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:34.150690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e100d-4557-4804-b2b4-c79d49b365a9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:34.162022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 396/768] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 13 08:48:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:34.163255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e100d-4557-4804-b2b4-c79d49b365a9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:34.276010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e100d-4557-4804-b2b4-c79d49b365a9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:34.304114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:34.318895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b0dd0-2042-43fd-aa4c-4ef4d0399ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:34.320019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b0dd0-2042-43fd-aa4c-4ef4d0399ecf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.321212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b0dd0-2042-43fd-aa4c-4ef4d0399ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.322210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b0dd0-2042-43fd-aa4c-4ef4d0399ecf None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.324983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b520f-a64b-4f0f-a15e-8cae2455cd5c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:34.325303 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b0dd0-2042-43fd-aa4c-4ef4d0399ecf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:34.325791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b0dd0-2042-43fd-aa4c-4ef4d0399ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:34.328776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b0dd0-2042-43fd-aa4c-4ef4d0399ecf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:34.342098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 397/769] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 13 08:48:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:34.349544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b0dd0-2042-43fd-aa4c-4ef4d0399ecf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:34.414838 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10853fd9-6049-4bc7-b2f3-d3e920f588ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:34.415428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853fd9-6049-4bc7-b2f3-d3e920f588ad None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.415711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853fd9-6049-4bc7-b2f3-d3e920f588ad None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.415988 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853fd9-6049-4bc7-b2f3-d3e920f588ad None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.416781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10853fd9-6049-4bc7-b2f3-d3e920f588ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:34.417089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10853fd9-6049-4bc7-b2f3-d3e920f588ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:34.418561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10853fd9-6049-4bc7-b2f3-d3e920f588ad None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:34.447580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 373/770] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:34.452923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10853fd9-6049-4bc7-b2f3-d3e920f588ad None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:34.471644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b0dd0-2042-43fd-aa4c-4ef4d0399ecf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:34.484254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:34.509716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d7ef9-815a-4802-a23f-ad7b6ad2ab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:34.510411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d7ef9-815a-4802-a23f-ad7b6ad2ab2f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.510696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d7ef9-815a-4802-a23f-ad7b6ad2ab2f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.510972 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d7ef9-815a-4802-a23f-ad7b6ad2ab2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.511769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d7ef9-815a-4802-a23f-ad7b6ad2ab2f None admin] RBAC: Adding query filter params (interface=public, service_id=cea4a58cce5546788e56dbebe224cfa9, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:34.512080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d7ef9-815a-4802-a23f-ad7b6ad2ab2f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:34.513459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d7ef9-815a-4802-a23f-ad7b6ad2ab2f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:34.521681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 398/771] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 13 08:48:34 2022] GET /identity/v3/endpoints?service_id=cea4a58cce5546788e56dbebe224cfa9&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:34.528033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d7ef9-815a-4802-a23f-ad7b6ad2ab2f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:34.593916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10853fd9-6049-4bc7-b2f3-d3e920f588ad None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:34.614522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:34.632026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427606c1-abb4-4a90-ad2a-86a5faade3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:34.633102 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427606c1-abb4-4a90-ad2a-86a5faade3c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.633382 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427606c1-abb4-4a90-ad2a-86a5faade3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.633660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427606c1-abb4-4a90-ad2a-86a5faade3c3 None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/8af26e036f4e466ebb923771c1f5f502/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.636432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427606c1-abb4-4a90-ad2a-86a5faade3c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=8af26e036f4e466ebb923771c1f5f502, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:34.638448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427606c1-abb4-4a90-ad2a-86a5faade3c3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:34.672529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d7ef9-815a-4802-a23f-ad7b6ad2ab2f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:34.708700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e299785-60e0-403c-b321-d2d39d20f4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:34.709680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e299785-60e0-403c-b321-d2d39d20f4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.710190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e299785-60e0-403c-b321-d2d39d20f4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.710647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e299785-60e0-403c-b321-d2d39d20f4a4 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.711471 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e299785-60e0-403c-b321-d2d39d20f4a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:34.713012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e299785-60e0-403c-b321-d2d39d20f4a4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:34.719127 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 399/772] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 13 08:48:34 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:34.721229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166b4da-3dd0-4357-81ba-c8e20408b340 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.721838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166b4da-3dd0-4357-81ba-c8e20408b340 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.722298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166b4da-3dd0-4357-81ba-c8e20408b340 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.724151 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 400/774] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:34.724322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 374/774] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:48:34 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/8af26e036f4e466ebb923771c1f5f502/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 08:48:34.729228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f116a1-4b3d-4dd0-b8fe-0e892bec23ef None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.729761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f116a1-4b3d-4dd0-b8fe-0e892bec23ef None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.730281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f116a1-4b3d-4dd0-b8fe-0e892bec23ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.754679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-09f116a1-4b3d-4dd0-b8fe-0e892bec23ef None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:34.756339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-09f116a1-4b3d-4dd0-b8fe-0e892bec23ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:34.760843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc80f2-a24c-42cb-9157-e0e7217ceae5 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.761380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc80f2-a24c-42cb-9157-e0e7217ceae5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.761457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc80f2-a24c-42cb-9157-e0e7217ceae5 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.763300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 375/775] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:34.771528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e97626-60b5-43fa-97ec-cf1979c92623 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:34.771865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e97626-60b5-43fa-97ec-cf1979c92623 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:34.772094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e97626-60b5-43fa-97ec-cf1979c92623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:34.796432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-21e97626-60b5-43fa-97ec-cf1979c92623 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:34.798173 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-21e97626-60b5-43fa-97ec-cf1979c92623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:34.844516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 401/776] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:34.884417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f116a1-4b3d-4dd0-b8fe-0e892bec23ef None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:34.887020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 376/777] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:34.916307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e97626-60b5-43fa-97ec-cf1979c92623 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:34.918539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:34.963986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:35.045513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f116a1-4b3d-4dd0-b8fe-0e892bec23ef None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:35.053999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e97626-60b5-43fa-97ec-cf1979c92623 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:35.086965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a29c4-cfb6-4c76-b510-e1a5e657c91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:35.087914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a29c4-cfb6-4c76-b510-e1a5e657c91d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.088270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a29c4-cfb6-4c76-b510-e1a5e657c91d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.088819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a29c4-cfb6-4c76-b510-e1a5e657c91d None admin] PATH_INFO: `/v3/services/network` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.089878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a29c4-cfb6-4c76-b510-e1a5e657c91d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:35.091764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a29c4-cfb6-4c76-b510-e1a5e657c91d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:35.099578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c186ff-4245-470f-95fd-c69968097acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:35.100638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c186ff-4245-470f-95fd-c69968097acb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.101826 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c186ff-4245-470f-95fd-c69968097acb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.102529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c186ff-4245-470f-95fd-c69968097acb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f09a29c4-cfb6-4c76-b510-e1a5e657c91d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:35.111541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:35.112753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 08:48:35.113778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:35.123045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 402/778] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:34 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:35.127479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c186ff-4245-470f-95fd-c69968097acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:35.134481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a29c4-cfb6-4c76-b510-e1a5e657c91d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-08c186ff-4245-470f-95fd-c69968097acb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:35.136163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:35.137284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:35.138449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 377/779] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:35.139293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024dd6b-2a8c-46ac-9feb-00cec5af1a5a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.139293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024dd6b-2a8c-46ac-9feb-00cec5af1a5a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.139438 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024dd6b-2a8c-46ac-9feb-00cec5af1a5a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.145611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 378/780] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:35.146363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8d538-ba7a-4a65-8fae-77b0a2ea39a1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.146649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8d538-ba7a-4a65-8fae-77b0a2ea39a1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.146927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8d538-ba7a-4a65-8fae-77b0a2ea39a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.169028 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1cf8d538-ba7a-4a65-8fae-77b0a2ea39a1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:35.170859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1cf8d538-ba7a-4a65-8fae-77b0a2ea39a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:35.248825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:35.249061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a29c4-cfb6-4c76-b510-e1a5e657c91d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:35.261267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:35.266113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 379/781] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:35.267166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8d538-ba7a-4a65-8fae-77b0a2ea39a1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:35.306426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a2d59-dff6-488d-a5f0-d97c163c6276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:35.307333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a2d59-dff6-488d-a5f0-d97c163c6276 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.307621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a2d59-dff6-488d-a5f0-d97c163c6276 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.307896 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a2d59-dff6-488d-a5f0-d97c163c6276 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.309014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a2d59-dff6-488d-a5f0-d97c163c6276 None admin] RBAC: Adding query filter params (name=network) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:35.309412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a2d59-dff6-488d-a5f0-d97c163c6276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:35.311330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a2d59-dff6-488d-a5f0-d97c163c6276 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:35.318647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 403/782] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:48:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:35.320702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a2d59-dff6-488d-a5f0-d97c163c6276 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:35.413106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8d538-ba7a-4a65-8fae-77b0a2ea39a1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:35.439318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a2d59-dff6-488d-a5f0-d97c163c6276 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:35.460207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41fb3-c244-4686-b3b9-460213105f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:35.461233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41fb3-c244-4686-b3b9-460213105f4e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.461711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41fb3-c244-4686-b3b9-460213105f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.462735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41fb3-c244-4686-b3b9-460213105f4e None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.463948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41fb3-c244-4686-b3b9-460213105f4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:35.464452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41fb3-c244-4686-b3b9-460213105f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:35.466890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41fb3-c244-4686-b3b9-460213105f4e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:35.476710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043fdad-d13d-4a55-bd9f-258532db4f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:35.477565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043fdad-d13d-4a55-bd9f-258532db4f63 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.477852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043fdad-d13d-4a55-bd9f-258532db4f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.478169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043fdad-d13d-4a55-bd9f-258532db4f63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.479327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 380/783] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:35.480467 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41fb3-c244-4686-b3b9-460213105f4e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:35.494558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043fdad-d13d-4a55-bd9f-258532db4f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8043fdad-d13d-4a55-bd9f-258532db4f63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:35.503050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:35.503961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:35.504808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:35.509144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 404/784] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:35.509988 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043fdad-d13d-4a55-bd9f-258532db4f63 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:35.550403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:35.561071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:35.622410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41fb3-c244-4686-b3b9-460213105f4e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:35.623590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043fdad-d13d-4a55-bd9f-258532db4f63 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:35.662057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02002a1c-37f3-414b-95f8-57e1f51c0054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:35.663000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02002a1c-37f3-414b-95f8-57e1f51c0054 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.663283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02002a1c-37f3-414b-95f8-57e1f51c0054 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.663554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02002a1c-37f3-414b-95f8-57e1f51c0054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.668897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa00f0f-6217-42c3-b12b-582710f81fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:35.670058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00f0f-6217-42c3-b12b-582710f81fcd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.670537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00f0f-6217-42c3-b12b-582710f81fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.671035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00f0f-6217-42c3-b12b-582710f81fcd None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.672462 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa00f0f-6217-42c3-b12b-582710f81fcd None admin] RBAC: Adding query filter params (type=network) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:35.673015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa00f0f-6217-42c3-b12b-582710f81fcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:35.675351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa00f0f-6217-42c3-b12b-582710f81fcd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:35.675538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02002a1c-37f3-414b-95f8-57e1f51c0054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:35.677053 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02002a1c-37f3-414b-95f8-57e1f51c0054 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:35.682268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 381/785] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:48:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:35.683349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa00f0f-6217-42c3-b12b-582710f81fcd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-02002a1c-37f3-414b-95f8-57e1f51c0054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:35.697279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:35.698253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:35.699138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:35.699138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:35.699138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:35.699138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:35.703479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 405/786] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:35.705205 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02002a1c-37f3-414b-95f8-57e1f51c0054 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:35.803541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:35.808613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa00f0f-6217-42c3-b12b-582710f81fcd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:35.823396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02002a1c-37f3-414b-95f8-57e1f51c0054 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:35.827376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:35.844940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcf0e2-e771-49a9-be74-ba64f7ec0bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:35.845740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcf0e2-e771-49a9-be74-ba64f7ec0bac None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.846166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcf0e2-e771-49a9-be74-ba64f7ec0bac None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.846544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcf0e2-e771-49a9-be74-ba64f7ec0bac None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.847383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcf0e2-e771-49a9-be74-ba64f7ec0bac None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:35.847788 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcf0e2-e771-49a9-be74-ba64f7ec0bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:35.849643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcf0e2-e771-49a9-be74-ba64f7ec0bac None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:35.865032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 382/787] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:35.865472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9b593-23e0-41a1-b4ac-ecdc889cc1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:35.866337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9b593-23e0-41a1-b4ac-ecdc889cc1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:35.866452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcf0e2-e771-49a9-be74-ba64f7ec0bac None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:35.866925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9b593-23e0-41a1-b4ac-ecdc889cc1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:35.867338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9b593-23e0-41a1-b4ac-ecdc889cc1f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:35.868249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9b593-23e0-41a1-b4ac-ecdc889cc1f9 None admin] RBAC: Adding query filter params (interface=public, service_id=694536705d01402996db5a7c62022c68, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:35.868715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9b593-23e0-41a1-b4ac-ecdc889cc1f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:35.871264 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9b593-23e0-41a1-b4ac-ecdc889cc1f9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:35.878012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 406/788] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 13 08:48:35 2022] GET /identity/v3/endpoints?service_id=694536705d01402996db5a7c62022c68&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:35.883248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9b593-23e0-41a1-b4ac-ecdc889cc1f9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:35.988063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcf0e2-e771-49a9-be74-ba64f7ec0bac None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:36.007015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9b593-23e0-41a1-b4ac-ecdc889cc1f9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:36.027603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030abd00-697d-4935-b288-6a2ab582ce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:36.028303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030abd00-697d-4935-b288-6a2ab582ce44 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:36.028564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030abd00-697d-4935-b288-6a2ab582ce44 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:36.028839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030abd00-697d-4935-b288-6a2ab582ce44 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:36.029565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030abd00-697d-4935-b288-6a2ab582ce44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:36.029873 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030abd00-697d-4935-b288-6a2ab582ce44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:36.031523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030abd00-697d-4935-b288-6a2ab582ce44 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:36.052608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 383/789] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:36.052608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030abd00-697d-4935-b288-6a2ab582ce44 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:36.057577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012f4c9-9d35-4639-80e4-3e8717e806ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:36.058516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012f4c9-9d35-4639-80e4-3e8717e806ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:36.058895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012f4c9-9d35-4639-80e4-3e8717e806ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:36.059370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012f4c9-9d35-4639-80e4-3e8717e806ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:36.073704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012f4c9-9d35-4639-80e4-3e8717e806ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:36.075587 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012f4c9-9d35-4639-80e4-3e8717e806ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3012f4c9-9d35-4639-80e4-3e8717e806ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:36.097299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:36.098692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:36.099657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:36.099657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:36.099657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:36.099657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:36.099875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:36.104351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 407/790] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:36.105272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012f4c9-9d35-4639-80e4-3e8717e806ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:36.108378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:36.159996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030abd00-697d-4935-b288-6a2ab582ce44 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:36.195721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16cf5-2c1b-4824-bb57-e891dc8d8a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:36.196319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16cf5-2c1b-4824-bb57-e891dc8d8a72 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:36.196603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16cf5-2c1b-4824-bb57-e891dc8d8a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:36.196878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16cf5-2c1b-4824-bb57-e891dc8d8a72 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:36.197562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a16cf5-2c1b-4824-bb57-e891dc8d8a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:36.198971 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a16cf5-2c1b-4824-bb57-e891dc8d8a72 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:36.204969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 384/791] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 13 08:48:36 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:36.205856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16cf5-2c1b-4824-bb57-e891dc8d8a72 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:36.222116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012f4c9-9d35-4639-80e4-3e8717e806ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:36.262986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67610c-607e-4da3-b2db-590b98ce485a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:36.264137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67610c-607e-4da3-b2db-590b98ce485a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:36.264595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67610c-607e-4da3-b2db-590b98ce485a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:36.265013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67610c-607e-4da3-b2db-590b98ce485a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:36.274779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67610c-607e-4da3-b2db-590b98ce485a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:36.276424 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67610c-607e-4da3-b2db-590b98ce485a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2d67610c-607e-4da3-b2db-590b98ce485a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:36.292797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:36.293986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:36.295391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:36.295391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:36.295391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:36.295391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:36.295391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:36.295391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:36.295391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:36.295391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:36.295391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:36.295391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:36.297666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 408/792] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:36.304220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67610c-607e-4da3-b2db-590b98ce485a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:36.316399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16cf5-2c1b-4824-bb57-e891dc8d8a72 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:36.318956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:36.364937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf685f5-95dc-4d23-aed8-ea1386e154d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:36.365916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf685f5-95dc-4d23-aed8-ea1386e154d0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:36.366229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf685f5-95dc-4d23-aed8-ea1386e154d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:36.366504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf685f5-95dc-4d23-aed8-ea1386e154d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:36.367263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf685f5-95dc-4d23-aed8-ea1386e154d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:36.367571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf685f5-95dc-4d23-aed8-ea1386e154d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:36.369202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf685f5-95dc-4d23-aed8-ea1386e154d0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:36.378721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:36.398575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 385/793] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:36.405153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf685f5-95dc-4d23-aed8-ea1386e154d0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:36.429130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67610c-607e-4da3-b2db-590b98ce485a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:36.476293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaa371-8c44-4c6c-9bc3-8e45ab9d0304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:36.477364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaa371-8c44-4c6c-9bc3-8e45ab9d0304 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:36.477647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaa371-8c44-4c6c-9bc3-8e45ab9d0304 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:36.478205 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eaa371-8c44-4c6c-9bc3-8e45ab9d0304 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:36.480495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eaa371-8c44-4c6c-9bc3-8e45ab9d0304 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:36.480804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eaa371-8c44-4c6c-9bc3-8e45ab9d0304 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:36.483068 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eaa371-8c44-4c6c-9bc3-8e45ab9d0304 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:36.509027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 409/794] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 13 08:48:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:36.515592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaa371-8c44-4c6c-9bc3-8e45ab9d0304 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:36.553975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf685f5-95dc-4d23-aed8-ea1386e154d0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:36.602527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61947-d802-4802-8499-ab53fead9e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:36.605185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61947-d802-4802-8499-ab53fead9e5b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:36.605713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61947-d802-4802-8499-ab53fead9e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:36.606301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61947-d802-4802-8499-ab53fead9e5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:36.615652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db61947-d802-4802-8499-ab53fead9e5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:36.617540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db61947-d802-4802-8499-ab53fead9e5b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7db61947-d802-4802-8499-ab53fead9e5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:36.638097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:36.639120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:36.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:36.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:36.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:36.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:36.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:36.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:36.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:36.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:36.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:36.640069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:36.642479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eaa371-8c44-4c6c-9bc3-8e45ab9d0304 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:36.644381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 386/795] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 13 08:48:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:36.650022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61947-d802-4802-8499-ab53fead9e5b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:36.666543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:36.687687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89aa754-2036-45db-90bb-e1132364a320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:36.689588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89aa754-2036-45db-90bb-e1132364a320 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:36.692470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89aa754-2036-45db-90bb-e1132364a320 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:36.694256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89aa754-2036-45db-90bb-e1132364a320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:36.711981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:36.720275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89aa754-2036-45db-90bb-e1132364a320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:36.721759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89aa754-2036-45db-90bb-e1132364a320 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b89aa754-2036-45db-90bb-e1132364a320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:36.767158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:36.768326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:36.769507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:36.769507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:36.769507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:36.769507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:36.771170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 410/796] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:36.776698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89aa754-2036-45db-90bb-e1132364a320 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:36.846143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61947-d802-4802-8499-ab53fead9e5b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:36.924392 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d0355-32c1-477c-940a-fc7713b4d79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:36.925073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d0355-32c1-477c-940a-fc7713b4d79b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:36.926061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d0355-32c1-477c-940a-fc7713b4d79b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:36.926351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d0355-32c1-477c-940a-fc7713b4d79b None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:36.928547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d0355-32c1-477c-940a-fc7713b4d79b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:36.928860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d0355-32c1-477c-940a-fc7713b4d79b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:36.931237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d0355-32c1-477c-940a-fc7713b4d79b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:36.961044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89aa754-2036-45db-90bb-e1132364a320 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:36.971836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 387/797] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 13 08:48:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:36.980101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d0355-32c1-477c-940a-fc7713b4d79b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:37.030734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816bbdb-d2bb-4fc7-b3e9-9f6a62e5020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:37.031782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816bbdb-d2bb-4fc7-b3e9-9f6a62e5020b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.032818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816bbdb-d2bb-4fc7-b3e9-9f6a62e5020b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.033303 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816bbdb-d2bb-4fc7-b3e9-9f6a62e5020b None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.034300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816bbdb-d2bb-4fc7-b3e9-9f6a62e5020b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:37.034818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816bbdb-d2bb-4fc7-b3e9-9f6a62e5020b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:37.036420 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816bbdb-d2bb-4fc7-b3e9-9f6a62e5020b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:37.057569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 411/798] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:37.067035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816bbdb-d2bb-4fc7-b3e9-9f6a62e5020b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:37.082777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:37.092205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d0355-32c1-477c-940a-fc7713b4d79b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:37.118450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:37.130455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58691f8-65f4-431e-9dbe-cdabc5124046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:37.131382 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58691f8-65f4-431e-9dbe-cdabc5124046 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.131667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58691f8-65f4-431e-9dbe-cdabc5124046 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.131939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58691f8-65f4-431e-9dbe-cdabc5124046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.142792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58691f8-65f4-431e-9dbe-cdabc5124046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:37.144244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58691f8-65f4-431e-9dbe-cdabc5124046 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-a58691f8-65f4-431e-9dbe-cdabc5124046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:37.163635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:37.164776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:37.165672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:37.165672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:37.165672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:37.165672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:37.166377 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 388/799] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:37.177513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b15e7-08b0-4c64-a4a7-086e9893ad79 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.177513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b15e7-08b0-4c64-a4a7-086e9893ad79 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.177513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b15e7-08b0-4c64-a4a7-086e9893ad79 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.177513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 389/800] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:37.177513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b944f-ff0d-482a-92cb-6ed37d341249 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.177513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b944f-ff0d-482a-92cb-6ed37d341249 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.177963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b944f-ff0d-482a-92cb-6ed37d341249 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.177963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 390/801] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:37.178345 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b944f-ff0d-482a-92cb-6ed37d341249 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:37.195292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816bbdb-d2bb-4fc7-b3e9-9f6a62e5020b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:37.231628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b8ef6-e6be-439d-970b-41accce5d06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:37.232428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b8ef6-e6be-439d-970b-41accce5d06e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.232713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b8ef6-e6be-439d-970b-41accce5d06e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.232989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b8ef6-e6be-439d-970b-41accce5d06e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.240814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b8ef6-e6be-439d-970b-41accce5d06e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:37.242339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b8ef6-e6be-439d-970b-41accce5d06e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-af1b8ef6-e6be-439d-970b-41accce5d06e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:37.256457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:37.257376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 412/802] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e155a-3dd8-42ff-acda-67820af7a66a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.259142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e155a-3dd8-42ff-acda-67820af7a66a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.259771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e155a-3dd8-42ff-acda-67820af7a66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.276229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-168e155a-3dd8-42ff-acda-67820af7a66a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:37.277409 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-168e155a-3dd8-42ff-acda-67820af7a66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:37.293149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b944f-ff0d-482a-92cb-6ed37d341249 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:37.332321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9edc0b-5381-4692-b890-740aaa03b71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:37.332861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9edc0b-5381-4692-b890-740aaa03b71a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.333142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9edc0b-5381-4692-b890-740aaa03b71a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.333415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9edc0b-5381-4692-b890-740aaa03b71a None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.334166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9edc0b-5381-4692-b890-740aaa03b71a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:37.334475 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9edc0b-5381-4692-b890-740aaa03b71a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:37.335837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9edc0b-5381-4692-b890-740aaa03b71a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:37.353100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 391/803] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:37.353100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d7f3-c54c-4508-a556-c83faee6fb72 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.353100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d7f3-c54c-4508-a556-c83faee6fb72 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.353100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d7f3-c54c-4508-a556-c83faee6fb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.355545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:37.366128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 413/804] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:37.366128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e155a-3dd8-42ff-acda-67820af7a66a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:37.368930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3538d7f3-c54c-4508-a556-c83faee6fb72 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:37.370206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3538d7f3-c54c-4508-a556-c83faee6fb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:37.394117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:37.439164 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 392/805] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:37.450868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538d7f3-c54c-4508-a556-c83faee6fb72 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:37.476775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e155a-3dd8-42ff-acda-67820af7a66a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:37.512713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6537bf-da42-4c9a-8213-4aa494de784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:37.513364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6537bf-da42-4c9a-8213-4aa494de784f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.513645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6537bf-da42-4c9a-8213-4aa494de784f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.513920 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6537bf-da42-4c9a-8213-4aa494de784f None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.514724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6537bf-da42-4c9a-8213-4aa494de784f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:37.515030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6537bf-da42-4c9a-8213-4aa494de784f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:37.516486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6537bf-da42-4c9a-8213-4aa494de784f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:37.530156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 414/806] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:37.530156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6537bf-da42-4c9a-8213-4aa494de784f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:37.562815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538d7f3-c54c-4508-a556-c83faee6fb72 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:37.588391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:37.599589 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf833d6-6d9f-4976-bab1-de3991848466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:37.600734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf833d6-6d9f-4976-bab1-de3991848466 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.600734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf833d6-6d9f-4976-bab1-de3991848466 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.600974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf833d6-6d9f-4976-bab1-de3991848466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.607524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf833d6-6d9f-4976-bab1-de3991848466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:37.608983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf833d6-6d9f-4976-bab1-de3991848466 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:37.619277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-faf833d6-6d9f-4976-bab1-de3991848466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:37.623032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:37.623981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:37.624878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:37.624878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:37.624878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:37.624878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:37.624878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:37.624878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:37.624878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:37.624878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:37.629653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 393/807] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:37.630446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf833d6-6d9f-4976-bab1-de3991848466 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:37.641695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6537bf-da42-4c9a-8213-4aa494de784f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:37.678354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1494fc-f498-45b1-b47d-1d118caa77ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:37.679289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1494fc-f498-45b1-b47d-1d118caa77ca None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.679572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1494fc-f498-45b1-b47d-1d118caa77ca None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.679848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1494fc-f498-45b1-b47d-1d118caa77ca None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.680513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1494fc-f498-45b1-b47d-1d118caa77ca None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:37.682085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1494fc-f498-45b1-b47d-1d118caa77ca None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ed1494fc-f498-45b1-b47d-1d118caa77ca None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:37.696487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:37.697459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:37.698361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 415/808] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Jul 13 08:48:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:37.699208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1494fc-f498-45b1-b47d-1d118caa77ca None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:37.739014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf833d6-6d9f-4976-bab1-de3991848466 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:37.774443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a98e6-7071-4d3e-8ad3-297fae3e9e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:37.775227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a98e6-7071-4d3e-8ad3-297fae3e9e22 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.775508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a98e6-7071-4d3e-8ad3-297fae3e9e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.775783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a98e6-7071-4d3e-8ad3-297fae3e9e22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.785870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a98e6-7071-4d3e-8ad3-297fae3e9e22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-a65a98e6-7071-4d3e-8ad3-297fae3e9e22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:37.793389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:37.794348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:37.795436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 394/809] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:37.796130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a98e6-7071-4d3e-8ad3-297fae3e9e22 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:37.814539 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1494fc-f498-45b1-b47d-1d118caa77ca None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:37.847003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:37.853036 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2b12b-db85-430b-8960-e7a14965fa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:37.853927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b12b-db85-430b-8960-e7a14965fa0f None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.854249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b12b-db85-430b-8960-e7a14965fa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.854519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b12b-db85-430b-8960-e7a14965fa0f None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/906fc971a5ee4ffd9503ab13d46cd605/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.856508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2b12b-db85-430b-8960-e7a14965fa0f None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=906fc971a5ee4ffd9503ab13d46cd605, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:37.857953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2b12b-db85-430b-8960-e7a14965fa0f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:37.901467 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a98e6-7071-4d3e-8ad3-297fae3e9e22 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:37.935982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 416/810] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:48:37 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/906fc971a5ee4ffd9503ab13d46cd605/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:48:37.936122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1752a7-aa92-4880-a9c4-9d2c7132dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:37.936786 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1752a7-aa92-4880-a9c4-9d2c7132dfda None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:37.937072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1752a7-aa92-4880-a9c4-9d2c7132dfda None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:37.937351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1752a7-aa92-4880-a9c4-9d2c7132dfda None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:37.937622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2b12b-db85-430b-8960-e7a14965fa0f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:37.938186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1752a7-aa92-4880-a9c4-9d2c7132dfda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:37.939035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1752a7-aa92-4880-a9c4-9d2c7132dfda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:37.940608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1752a7-aa92-4880-a9c4-9d2c7132dfda None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:37.959304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:37.965115 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 395/811] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:37.966170 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1752a7-aa92-4880-a9c4-9d2c7132dfda None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:38.086567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2b12b-db85-430b-8960-e7a14965fa0f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:38.105264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1752a7-aa92-4880-a9c4-9d2c7132dfda None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:38.107999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:38.134132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2ab57-9746-4aea-9774-783e133f6f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:38.134827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2ab57-9746-4aea-9774-783e133f6f81 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:38.135272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2ab57-9746-4aea-9774-783e133f6f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:38.135558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2ab57-9746-4aea-9774-783e133f6f81 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:38.136611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2ab57-9746-4aea-9774-783e133f6f81 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:38.137192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2ab57-9746-4aea-9774-783e133f6f81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:38.139104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2ab57-9746-4aea-9774-783e133f6f81 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:38.145449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 417/812] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 13 08:48:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:38.146300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2ab57-9746-4aea-9774-783e133f6f81 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:38.153289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408272a-93ad-4519-bcc9-e2f1f199db2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:38.153289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408272a-93ad-4519-bcc9-e2f1f199db2b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:38.153499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408272a-93ad-4519-bcc9-e2f1f199db2b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:38.153748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408272a-93ad-4519-bcc9-e2f1f199db2b None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:38.154603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408272a-93ad-4519-bcc9-e2f1f199db2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:38.154911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408272a-93ad-4519-bcc9-e2f1f199db2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:38.156276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408272a-93ad-4519-bcc9-e2f1f199db2b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:38.166002 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 396/813] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:38.167176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408272a-93ad-4519-bcc9-e2f1f199db2b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:38.240349 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:38.273376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408272a-93ad-4519-bcc9-e2f1f199db2b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:38.276506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2ab57-9746-4aea-9774-783e133f6f81 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:38.308939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982a078-91b0-4df9-90eb-ee8fe006e86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:38.309882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982a078-91b0-4df9-90eb-ee8fe006e86e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:38.310172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982a078-91b0-4df9-90eb-ee8fe006e86e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:38.310545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982a078-91b0-4df9-90eb-ee8fe006e86e None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:38.311355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982a078-91b0-4df9-90eb-ee8fe006e86e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:38.311761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982a078-91b0-4df9-90eb-ee8fe006e86e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:38.314012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982a078-91b0-4df9-90eb-ee8fe006e86e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:38.316539 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c71e65-05c1-4f36-8902-1cf8f91ccf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:38.317593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c71e65-05c1-4f36-8902-1cf8f91ccf50 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:38.318009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c71e65-05c1-4f36-8902-1cf8f91ccf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:38.318474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c71e65-05c1-4f36-8902-1cf8f91ccf50 None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/0b380bc85da64a8bb687e2e5c4577e79/roles/3ec600afe034440d873fe4834644b63b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:38.324622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 397/814] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 13 08:48:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:38.325610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982a078-91b0-4df9-90eb-ee8fe006e86e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:38.337218 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c71e65-05c1-4f36-8902-1cf8f91ccf50 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=0b380bc85da64a8bb687e2e5c4577e79, role_id=3ec600afe034440d873fe4834644b63b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:38.339104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c71e65-05c1-4f36-8902-1cf8f91ccf50 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:38.358398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:38.406417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 418/815] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:48:38 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/0b380bc85da64a8bb687e2e5c4577e79/roles/3ec600afe034440d873fe4834644b63b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:48:38.409786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c71e65-05c1-4f36-8902-1cf8f91ccf50 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:38.481102 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982a078-91b0-4df9-90eb-ee8fe006e86e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:38.523755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96acc9-fb59-4ef6-8371-3fc28baf32a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:38.524975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96acc9-fb59-4ef6-8371-3fc28baf32a2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:38.525453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96acc9-fb59-4ef6-8371-3fc28baf32a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:38.525919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96acc9-fb59-4ef6-8371-3fc28baf32a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:38.538689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96acc9-fb59-4ef6-8371-3fc28baf32a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:38.540553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96acc9-fb59-4ef6-8371-3fc28baf32a2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-bb96acc9-fb59-4ef6-8371-3fc28baf32a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:38.561225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:38.562200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:38.563090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:38.563090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:38.563090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:38.563090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:38.566210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 398/816] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:38.577825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96acc9-fb59-4ef6-8371-3fc28baf32a2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:38.621060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c71e65-05c1-4f36-8902-1cf8f91ccf50 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:38.644836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:38.661189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b5333-8924-45f8-9521-7079f4701ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:38.661917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b5333-8924-45f8-9521-7079f4701ee9 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:38.662235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b5333-8924-45f8-9521-7079f4701ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:38.662515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b5333-8924-45f8-9521-7079f4701ee9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:38.663241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b5333-8924-45f8-9521-7079f4701ee9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:38.664731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b5333-8924-45f8-9521-7079f4701ee9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:38.694599 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:38.713621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96acc9-fb59-4ef6-8371-3fc28baf32a2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:38.747487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8895b3-6cf5-4291-88e7-29744205bcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:38.748138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8895b3-6cf5-4291-88e7-29744205bcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:38.748415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8895b3-6cf5-4291-88e7-29744205bcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:38.748735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8895b3-6cf5-4291-88e7-29744205bcb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:38.749448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8895b3-6cf5-4291-88e7-29744205bcb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:38.749748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8895b3-6cf5-4291-88e7-29744205bcb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:38.751980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8895b3-6cf5-4291-88e7-29744205bcb3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:38.762631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 419/817] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 13 08:48:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:38.763952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd0048-8212-4ec9-afeb-eb2592643daf None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:38.764238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd0048-8212-4ec9-afeb-eb2592643daf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:38.771897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd0048-8212-4ec9-afeb-eb2592643daf None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:38.771897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 420/818] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:38.773393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 399/819] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:38.776888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2383dd-dbd7-40dd-bf66-81726fc5e904 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:38.777454 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2383dd-dbd7-40dd-bf66-81726fc5e904 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:38.777943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2383dd-dbd7-40dd-bf66-81726fc5e904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:38.784429 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8895b3-6cf5-4291-88e7-29744205bcb3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:38.817210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1b2383dd-dbd7-40dd-bf66-81726fc5e904 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:38.819694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1b2383dd-dbd7-40dd-bf66-81726fc5e904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:38.963651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8895b3-6cf5-4291-88e7-29744205bcb3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:38.971709 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 421/820] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:38.998082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2383dd-dbd7-40dd-bf66-81726fc5e904 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:39.009612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da3504-8314-4560-8cfa-41827a9a9085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:39.010578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3504-8314-4560-8cfa-41827a9a9085 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:39.010958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3504-8314-4560-8cfa-41827a9a9085 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:39.011238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3504-8314-4560-8cfa-41827a9a9085 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:39.021742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da3504-8314-4560-8cfa-41827a9a9085 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:39.023247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da3504-8314-4560-8cfa-41827a9a9085 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-91da3504-8314-4560-8cfa-41827a9a9085 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:39.038817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:39.039815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:39.040736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 400/821] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 13 08:48:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:39.046843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da3504-8314-4560-8cfa-41827a9a9085 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:39.049761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:39.138180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2383dd-dbd7-40dd-bf66-81726fc5e904 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:39.140442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:39.180183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da3504-8314-4560-8cfa-41827a9a9085 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:39.185837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81c97d-eb2b-4bf0-a23f-36e088f4dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:39.186681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81c97d-eb2b-4bf0-a23f-36e088f4dc1d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:39.187814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81c97d-eb2b-4bf0-a23f-36e088f4dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:39.188105 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81c97d-eb2b-4bf0-a23f-36e088f4dc1d None admin] PATH_INFO: `/v3/services/network` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:39.189667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81c97d-eb2b-4bf0-a23f-36e088f4dc1d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:39.191093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81c97d-eb2b-4bf0-a23f-36e088f4dc1d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-cf81c97d-eb2b-4bf0-a23f-36e088f4dc1d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:39.206086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:39.207148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:39.207979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 422/822] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:38 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:39.213997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81c97d-eb2b-4bf0-a23f-36e088f4dc1d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:39.258626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4b32d-4b7d-4463-9b73-ef51f2a56a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:39.259710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4b32d-4b7d-4463-9b73-ef51f2a56a64 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:39.260203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4b32d-4b7d-4463-9b73-ef51f2a56a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:39.261746 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4b32d-4b7d-4463-9b73-ef51f2a56a64 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:39.262833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4b32d-4b7d-4463-9b73-ef51f2a56a64 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:39.265005 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4b32d-4b7d-4463-9b73-ef51f2a56a64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:39.268690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4b32d-4b7d-4463-9b73-ef51f2a56a64 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:39.294144 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 401/823] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 13 08:48:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:39.303047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4b32d-4b7d-4463-9b73-ef51f2a56a64 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:39.418686 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4b32d-4b7d-4463-9b73-ef51f2a56a64 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:39.427158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:39.456079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610ba4e-0f21-4a05-bbb7-c7e426418970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:39.456737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610ba4e-0f21-4a05-bbb7-c7e426418970 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:39.457021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610ba4e-0f21-4a05-bbb7-c7e426418970 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:39.457303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610ba4e-0f21-4a05-bbb7-c7e426418970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:39.488226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81c97d-eb2b-4bf0-a23f-36e088f4dc1d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:39.528573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b5d0e-4ab8-433c-b03c-418ac5c728ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:39.529248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b5d0e-4ab8-433c-b03c-418ac5c728ba None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:39.529533 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b5d0e-4ab8-433c-b03c-418ac5c728ba None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:39.529814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b5d0e-4ab8-433c-b03c-418ac5c728ba None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:39.530553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b5d0e-4ab8-433c-b03c-418ac5c728ba None admin] RBAC: Adding query filter params (name=network) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:39.531325 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b5d0e-4ab8-433c-b03c-418ac5c728ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:39.533549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b5d0e-4ab8-433c-b03c-418ac5c728ba None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:39.538654 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 423/824] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:48:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:39.543760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b5d0e-4ab8-433c-b03c-418ac5c728ba None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:39.574477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610ba4e-0f21-4a05-bbb7-c7e426418970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:39.576931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610ba4e-0f21-4a05-bbb7-c7e426418970 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:39.595559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:39.697778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:39.717123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b5d0e-4ab8-433c-b03c-418ac5c728ba None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:39.755774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b4e98-cc2b-4894-9733-4d8bf22e9ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:39.755774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b4e98-cc2b-4894-9733-4d8bf22e9ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:39.755774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b4e98-cc2b-4894-9733-4d8bf22e9ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:39.755774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b4e98-cc2b-4894-9733-4d8bf22e9ef7 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:39.756198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b4e98-cc2b-4894-9733-4d8bf22e9ef7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:39.756732 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b4e98-cc2b-4894-9733-4d8bf22e9ef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:39.758296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b4e98-cc2b-4894-9733-4d8bf22e9ef7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:39.763936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 424/825] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:48:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:39.769793 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b4e98-cc2b-4894-9733-4d8bf22e9ef7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:39.786535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 402/826] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:48:39.795298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610ba4e-0f21-4a05-bbb7-c7e426418970 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:39.947576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610ba4e-0f21-4a05-bbb7-c7e426418970 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:39.983503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785865b9-3a7e-4e1f-a441-1dcd04838a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:39.984314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785865b9-3a7e-4e1f-a441-1dcd04838a66 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:39.984599 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785865b9-3a7e-4e1f-a441-1dcd04838a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:39.984877 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785865b9-3a7e-4e1f-a441-1dcd04838a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:39.995519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785865b9-3a7e-4e1f-a441-1dcd04838a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:39.996965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785865b9-3a7e-4e1f-a441-1dcd04838a66 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:39.998782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:40.016158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b4e98-cc2b-4894-9733-4d8bf22e9ef7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-785865b9-3a7e-4e1f-a441-1dcd04838a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:40.023719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:40.024626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:40.025542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:40.025542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:40.025542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:40.025542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:40.025542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 403/827] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:40.031584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785865b9-3a7e-4e1f-a441-1dcd04838a66 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:40.062292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:40.086762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263002e0-ca1c-40e2-9807-5929f88764c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:40.087435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263002e0-ca1c-40e2-9807-5929f88764c5 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.087723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263002e0-ca1c-40e2-9807-5929f88764c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.088003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263002e0-ca1c-40e2-9807-5929f88764c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.088782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263002e0-ca1c-40e2-9807-5929f88764c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:40.090377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263002e0-ca1c-40e2-9807-5929f88764c5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:40.153588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785865b9-3a7e-4e1f-a441-1dcd04838a66 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:40.174527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 425/828] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 13 08:48:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:40.208766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287c385-cf52-49eb-953d-d2416bcf65ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:40.209372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287c385-cf52-49eb-953d-d2416bcf65ed None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.209651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287c385-cf52-49eb-953d-d2416bcf65ed None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.209924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287c385-cf52-49eb-953d-d2416bcf65ed None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.210696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287c385-cf52-49eb-953d-d2416bcf65ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:40.211000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287c385-cf52-49eb-953d-d2416bcf65ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:40.212379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287c385-cf52-49eb-953d-d2416bcf65ed None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:40.225996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 404/829] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:40.231200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263002e0-ca1c-40e2-9807-5929f88764c5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:40.386194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b61c5-3dc3-46ef-b034-42d4b5ca2fe6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.386532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b61c5-3dc3-46ef-b034-42d4b5ca2fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.386745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b61c5-3dc3-46ef-b034-42d4b5ca2fe6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.388351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 405/830] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:40.390187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263002e0-ca1c-40e2-9807-5929f88764c5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:40.395667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0505f-dd56-43fb-b7d4-fc60cd1112e2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.395954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0505f-dd56-43fb-b7d4-fc60cd1112e2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.396232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0505f-dd56-43fb-b7d4-fc60cd1112e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.414533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5cb0505f-dd56-43fb-b7d4-fc60cd1112e2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:40.415706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5cb0505f-dd56-43fb-b7d4-fc60cd1112e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:40.438080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bf3d4-5cdc-40bd-abd0-e421f7ea2b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:40.439326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bf3d4-5cdc-40bd-abd0-e421f7ea2b3e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.439808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bf3d4-5cdc-40bd-abd0-e421f7ea2b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.440281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bf3d4-5cdc-40bd-abd0-e421f7ea2b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.447622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bf3d4-5cdc-40bd-abd0-e421f7ea2b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:40.449300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bf3d4-5cdc-40bd-abd0-e421f7ea2b3e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-cb6bf3d4-5cdc-40bd-abd0-e421f7ea2b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:40.464638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:40.465611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:40.466561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:40.466561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:40.466561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:40.466561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:40.466561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:40.466561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:40.466561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:40.466561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:40.469028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 426/831] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:40.475023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bf3d4-5cdc-40bd-abd0-e421f7ea2b3e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:40.482861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:40.492977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 406/832] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:40.494544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:40.524520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0505f-dd56-43fb-b7d4-fc60cd1112e2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:40.611994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bf3d4-5cdc-40bd-abd0-e421f7ea2b3e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:40.651888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0505f-dd56-43fb-b7d4-fc60cd1112e2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:40.660066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56556e72-911d-4644-9ed0-2f7213dd9745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:40.660750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56556e72-911d-4644-9ed0-2f7213dd9745 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.661035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56556e72-911d-4644-9ed0-2f7213dd9745 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.661310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56556e72-911d-4644-9ed0-2f7213dd9745 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.662132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56556e72-911d-4644-9ed0-2f7213dd9745 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:40.662449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56556e72-911d-4644-9ed0-2f7213dd9745 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:40.663904 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56556e72-911d-4644-9ed0-2f7213dd9745 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:40.683071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 427/833] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:40.689003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56556e72-911d-4644-9ed0-2f7213dd9745 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:40.701009 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe17afe-b39f-4a98-b218-723512673937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:40.701849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe17afe-b39f-4a98-b218-723512673937 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.702172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe17afe-b39f-4a98-b218-723512673937 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.702445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe17afe-b39f-4a98-b218-723512673937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.713381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe17afe-b39f-4a98-b218-723512673937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-afe17afe-b39f-4a98-b218-723512673937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:40.721689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:40.722635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:40.723474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 407/834] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:40.728772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe17afe-b39f-4a98-b218-723512673937 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:40.763026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:40.792807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:40.794985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56556e72-911d-4644-9ed0-2f7213dd9745 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:40.832174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46330a-0175-406c-baa2-757066a3062d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:40.832845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46330a-0175-406c-baa2-757066a3062d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.833127 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46330a-0175-406c-baa2-757066a3062d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.833402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46330a-0175-406c-baa2-757066a3062d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.834268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46330a-0175-406c-baa2-757066a3062d None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=8af26e036f4e466ebb923771c1f5f502) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:40.834579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46330a-0175-406c-baa2-757066a3062d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:40.836034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46330a-0175-406c-baa2-757066a3062d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:40.838768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe17afe-b39f-4a98-b218-723512673937 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:40.850348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 428/835] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:40 2022] GET /identity/v3/role_assignments?user.id=8af26e036f4e466ebb923771c1f5f502&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:40.878896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f2040-ecec-43e6-98bd-8133280670e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:40.879588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f2040-ecec-43e6-98bd-8133280670e1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.879869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f2040-ecec-43e6-98bd-8133280670e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.880143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f2040-ecec-43e6-98bd-8133280670e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.880908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f2040-ecec-43e6-98bd-8133280670e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:40.881212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f2040-ecec-43e6-98bd-8133280670e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:40.882641 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f2040-ecec-43e6-98bd-8133280670e1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:40.893409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 408/836] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:40.899120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46330a-0175-406c-baa2-757066a3062d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:40.903179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6bcf9-32b0-432a-80d0-df420b5f38e2 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.903484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6bcf9-32b0-432a-80d0-df420b5f38e2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.903881 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6bcf9-32b0-432a-80d0-df420b5f38e2 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.906081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 409/837] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:40.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e252f-59be-4b49-a258-f24b16f10334 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:40.911914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e252f-59be-4b49-a258-f24b16f10334 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:40.912259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e252f-59be-4b49-a258-f24b16f10334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:40.932087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6f7e252f-59be-4b49-a258-f24b16f10334 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:40.933305 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6f7e252f-59be-4b49-a258-f24b16f10334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:41.017551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 410/838] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:41.022822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46330a-0175-406c-baa2-757066a3062d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:41.040856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e252f-59be-4b49-a258-f24b16f10334 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:41.062463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6340cee-fb5b-4034-b5e2-31303336b6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:41.063888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6340cee-fb5b-4034-b5e2-31303336b6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.064178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6340cee-fb5b-4034-b5e2-31303336b6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.064444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6340cee-fb5b-4034-b5e2-31303336b6b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.068135 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:41.079313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6340cee-fb5b-4034-b5e2-31303336b6b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:41.081249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6340cee-fb5b-4034-b5e2-31303336b6b0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a6340cee-fb5b-4034-b5e2-31303336b6b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:41.100511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:41.101458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:41.106963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:41.106963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:41.106963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:41.106963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:41.106963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 429/839] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:41.107239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:41.107312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6340cee-fb5b-4034-b5e2-31303336b6b0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:41.168512 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e252f-59be-4b49-a258-f24b16f10334 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:41.215604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cf65b-ab51-48c2-b29f-0f172463694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:41.216484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cf65b-ab51-48c2-b29f-0f172463694b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.216853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cf65b-ab51-48c2-b29f-0f172463694b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.217027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cf65b-ab51-48c2-b29f-0f172463694b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.222100 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6340cee-fb5b-4034-b5e2-31303336b6b0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:41.228634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131cf65b-ab51-48c2-b29f-0f172463694b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-131cf65b-ab51-48c2-b29f-0f172463694b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:41.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:41.239121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 411/840] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:41.245157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f7fa0-72f1-478e-a5bd-2604749e2d05 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.245929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f7fa0-72f1-478e-a5bd-2604749e2d05 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.245929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f7fa0-72f1-478e-a5bd-2604749e2d05 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.245929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 412/841] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:41.246418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f7fa0-72f1-478e-a5bd-2604749e2d05 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:41.259698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eedef5-dda9-4c3d-be81-035e92ce87e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:41.260426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eedef5-dda9-4c3d-be81-035e92ce87e9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.260728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eedef5-dda9-4c3d-be81-035e92ce87e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.260986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eedef5-dda9-4c3d-be81-035e92ce87e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.261713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eedef5-dda9-4c3d-be81-035e92ce87e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:41.262018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eedef5-dda9-4c3d-be81-035e92ce87e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:41.263414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eedef5-dda9-4c3d-be81-035e92ce87e9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:41.276497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 430/842] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:41.278072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c3010-199c-4671-8271-5d9ebcc7300c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.278506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c3010-199c-4671-8271-5d9ebcc7300c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.278779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c3010-199c-4671-8271-5d9ebcc7300c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.298401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a09c3010-199c-4671-8271-5d9ebcc7300c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:41.299927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a09c3010-199c-4671-8271-5d9ebcc7300c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:41.347645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:41.366202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f7fa0-72f1-478e-a5bd-2604749e2d05 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:41.378172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:41.382385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 431/843] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:41.383283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c3010-199c-4671-8271-5d9ebcc7300c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:41.403191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248d022-2911-472d-b5f6-da11945065d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:41.403899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248d022-2911-472d-b5f6-da11945065d9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.404181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248d022-2911-472d-b5f6-da11945065d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.404456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248d022-2911-472d-b5f6-da11945065d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.405312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248d022-2911-472d-b5f6-da11945065d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:41.405791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248d022-2911-472d-b5f6-da11945065d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:41.407683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248d022-2911-472d-b5f6-da11945065d9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:41.416420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 413/844] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:41.417272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248d022-2911-472d-b5f6-da11945065d9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:41.495243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c3010-199c-4671-8271-5d9ebcc7300c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:41.530019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248d022-2911-472d-b5f6-da11945065d9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:41.530202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887e420-0549-4208-94d5-5412676b6258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:41.530995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887e420-0549-4208-94d5-5412676b6258 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.531274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887e420-0549-4208-94d5-5412676b6258 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.531548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887e420-0549-4208-94d5-5412676b6258 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.538897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887e420-0549-4208-94d5-5412676b6258 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:41.540333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887e420-0549-4208-94d5-5412676b6258 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d887e420-0549-4208-94d5-5412676b6258 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:41.555664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:41.556603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 432/845] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:41.557546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887e420-0549-4208-94d5-5412676b6258 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:41.571139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec335f-25ea-43bd-8c12-858fd91a314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:41.571964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec335f-25ea-43bd-8c12-858fd91a314a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.572296 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec335f-25ea-43bd-8c12-858fd91a314a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.572571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec335f-25ea-43bd-8c12-858fd91a314a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.573338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec335f-25ea-43bd-8c12-858fd91a314a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:41.574827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec335f-25ea-43bd-8c12-858fd91a314a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-02ec335f-25ea-43bd-8c12-858fd91a314a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:41.588182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:41.589188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:41.590484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 414/846] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:41.591319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec335f-25ea-43bd-8c12-858fd91a314a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:41.599736 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:41.629919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:41.668268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887e420-0549-4208-94d5-5412676b6258 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:41.703714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec335f-25ea-43bd-8c12-858fd91a314a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:41.705304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f7a9-754b-4405-9708-f403e581f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:41.706133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f7a9-754b-4405-9708-f403e581f519 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.706417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f7a9-754b-4405-9708-f403e581f519 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.706694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f7a9-754b-4405-9708-f403e581f519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.717652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9f7a9-754b-4405-9708-f403e581f519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:41.719912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9f7a9-754b-4405-9708-f403e581f519 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-acd9f7a9-754b-4405-9708-f403e581f519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:41.737534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:41.738604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:41.739706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:41.739706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:41.739706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:41.739706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:41.742670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b94781-e5da-4c26-a17f-7e43f07cfb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:41.743120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 433/847] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:41.743670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94781-e5da-4c26-a17f-7e43f07cfb34 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.743924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f7a9-754b-4405-9708-f403e581f519 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:41.744244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94781-e5da-4c26-a17f-7e43f07cfb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.744673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94781-e5da-4c26-a17f-7e43f07cfb34 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.745640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b94781-e5da-4c26-a17f-7e43f07cfb34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:41.746134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b94781-e5da-4c26-a17f-7e43f07cfb34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:41.748869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b94781-e5da-4c26-a17f-7e43f07cfb34 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:41.770064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 415/848] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:41.771152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b94781-e5da-4c26-a17f-7e43f07cfb34 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:41.841816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:41.854793 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f7a9-754b-4405-9708-f403e581f519 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:41.867677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:41.874043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b94781-e5da-4c26-a17f-7e43f07cfb34 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:41.892081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8668c9-94ee-4adf-8090-48db4b60fd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:41.892741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8668c9-94ee-4adf-8090-48db4b60fd97 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.893025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8668c9-94ee-4adf-8090-48db4b60fd97 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.893301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8668c9-94ee-4adf-8090-48db4b60fd97 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.894006 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8668c9-94ee-4adf-8090-48db4b60fd97 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:41.894340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8668c9-94ee-4adf-8090-48db4b60fd97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:41.895734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8668c9-94ee-4adf-8090-48db4b60fd97 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:41.900258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 434/849] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:41.901038 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8668c9-94ee-4adf-8090-48db4b60fd97 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:41.914657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b77c8-5386-4710-b85c-7556f8ef6271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:41.915453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b77c8-5386-4710-b85c-7556f8ef6271 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:41.915742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b77c8-5386-4710-b85c-7556f8ef6271 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:41.916018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b77c8-5386-4710-b85c-7556f8ef6271 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:41.916802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b77c8-5386-4710-b85c-7556f8ef6271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:41.917114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b77c8-5386-4710-b85c-7556f8ef6271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:41.918621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b77c8-5386-4710-b85c-7556f8ef6271 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:41.936406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 416/850] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:41.937559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b77c8-5386-4710-b85c-7556f8ef6271 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:42.017651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8668c9-94ee-4adf-8090-48db4b60fd97 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:42.045525 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b77c8-5386-4710-b85c-7556f8ef6271 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:42.055060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc7aed-13a9-4623-97cb-e362da0b5665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:42.055704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc7aed-13a9-4623-97cb-e362da0b5665 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:42.055990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc7aed-13a9-4623-97cb-e362da0b5665 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:42.056260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc7aed-13a9-4623-97cb-e362da0b5665 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:42.081977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:42.086506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9c04a-7af6-4196-8c3b-2eedcd895191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:42.087008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9c04a-7af6-4196-8c3b-2eedcd895191 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:42.087338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9c04a-7af6-4196-8c3b-2eedcd895191 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:42.087580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9c04a-7af6-4196-8c3b-2eedcd895191 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:42.088481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9c04a-7af6-4196-8c3b-2eedcd895191 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:42.088703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9c04a-7af6-4196-8c3b-2eedcd895191 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:42.090101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9c04a-7af6-4196-8c3b-2eedcd895191 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:42.114061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 417/851] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:42.114061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9c04a-7af6-4196-8c3b-2eedcd895191 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:42.127817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:42.154516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc7aed-13a9-4623-97cb-e362da0b5665 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:42.156804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc7aed-13a9-4623-97cb-e362da0b5665 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:42.223353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9c04a-7af6-4196-8c3b-2eedcd895191 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:42.262568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af97a1b-7931-4c0b-92cf-0bcdae7bcc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:42.263463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af97a1b-7931-4c0b-92cf-0bcdae7bcc35 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:42.263744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af97a1b-7931-4c0b-92cf-0bcdae7bcc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:42.264017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af97a1b-7931-4c0b-92cf-0bcdae7bcc35 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:42.273598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af97a1b-7931-4c0b-92cf-0bcdae7bcc35 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:42.275094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af97a1b-7931-4c0b-92cf-0bcdae7bcc35 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-4af97a1b-7931-4c0b-92cf-0bcdae7bcc35 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:42.293624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:42.295148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:42.296177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 418/852] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 13 08:48:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:42.301974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af97a1b-7931-4c0b-92cf-0bcdae7bcc35 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:42.308893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 435/853] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:48:42.315236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc7aed-13a9-4623-97cb-e362da0b5665 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:42.390137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:42.405404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:42.421120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af97a1b-7931-4c0b-92cf-0bcdae7bcc35 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:42.450888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc7aed-13a9-4623-97cb-e362da0b5665 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:42.459541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509b215-4a2a-4d85-a582-f7a1ed002081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:42.460345 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509b215-4a2a-4d85-a582-f7a1ed002081 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:42.460641 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509b215-4a2a-4d85-a582-f7a1ed002081 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:42.460917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509b215-4a2a-4d85-a582-f7a1ed002081 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:42.461923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509b215-4a2a-4d85-a582-f7a1ed002081 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:42.462278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509b215-4a2a-4d85-a582-f7a1ed002081 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:42.463776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509b215-4a2a-4d85-a582-f7a1ed002081 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:42.487726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 419/854] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 13 08:48:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:42.489831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509b215-4a2a-4d85-a582-f7a1ed002081 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:42.491858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd787712-2c53-4cfa-9434-cd3a2dae4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:42.492918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd787712-2c53-4cfa-9434-cd3a2dae4280 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:42.493399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd787712-2c53-4cfa-9434-cd3a2dae4280 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:42.493877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd787712-2c53-4cfa-9434-cd3a2dae4280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:42.505155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd787712-2c53-4cfa-9434-cd3a2dae4280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:42.506921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd787712-2c53-4cfa-9434-cd3a2dae4280 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:42.606069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509b215-4a2a-4d85-a582-f7a1ed002081 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:42.644848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349830d3-2ab7-4fde-b01d-2eeb54072b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:42.645413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349830d3-2ab7-4fde-b01d-2eeb54072b58 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:42.645693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349830d3-2ab7-4fde-b01d-2eeb54072b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:42.645968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349830d3-2ab7-4fde-b01d-2eeb54072b58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-fd787712-2c53-4cfa-9434-cd3a2dae4280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:42.647846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:42.648722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:42.649624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:42.649624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:42.649624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:42.649624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:42.650406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:42.652105 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 436/855] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:42.661257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd787712-2c53-4cfa-9434-cd3a2dae4280 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:42.737305 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349830d3-2ab7-4fde-b01d-2eeb54072b58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:42.739063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349830d3-2ab7-4fde-b01d-2eeb54072b58 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:42.786773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd787712-2c53-4cfa-9434-cd3a2dae4280 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:42.789258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:42.825035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e9326-49f9-4b0e-9315-8e82347c1930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:42.825704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e9326-49f9-4b0e-9315-8e82347c1930 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:42.825993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e9326-49f9-4b0e-9315-8e82347c1930 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:42.826297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e9326-49f9-4b0e-9315-8e82347c1930 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:42.827024 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e9326-49f9-4b0e-9315-8e82347c1930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:42.827330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e9326-49f9-4b0e-9315-8e82347c1930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:42.829312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e9326-49f9-4b0e-9315-8e82347c1930 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:42.847989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 437/856] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:42.849121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e9326-49f9-4b0e-9315-8e82347c1930 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:42.872405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 420/857] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:48:42.873935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349830d3-2ab7-4fde-b01d-2eeb54072b58 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:42.910953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:42.966458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e9326-49f9-4b0e-9315-8e82347c1930 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:42.978901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349830d3-2ab7-4fde-b01d-2eeb54072b58 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:43.003749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2115f06-eef5-48a5-ac0c-555c616dd1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.004851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2115f06-eef5-48a5-ac0c-555c616dd1bf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.004982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2115f06-eef5-48a5-ac0c-555c616dd1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.005225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2115f06-eef5-48a5-ac0c-555c616dd1bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.012676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2115f06-eef5-48a5-ac0c-555c616dd1bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.014191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2115f06-eef5-48a5-ac0c-555c616dd1bf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.017120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee216ff5-e01f-4883-9720-7c00d623582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.018480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee216ff5-e01f-4883-9720-7c00d623582e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.018752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee216ff5-e01f-4883-9720-7c00d623582e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.019022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee216ff5-e01f-4883-9720-7c00d623582e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.029316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee216ff5-e01f-4883-9720-7c00d623582e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a2115f06-eef5-48a5-ac0c-555c616dd1bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:43.029523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:43.030584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:43.031477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:43.031477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:43.031477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:43.031477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:43.031477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:43.031477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:43.031477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:43.031477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:43.031477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 438/858] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:43.032389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27195b-93fc-4371-915b-a13350cb0ec8 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.032677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27195b-93fc-4371-915b-a13350cb0ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.032949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27195b-93fc-4371-915b-a13350cb0ec8 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.034344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 439/859] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:43.035586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee216ff5-e01f-4883-9720-7c00d623582e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.038744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27195b-93fc-4371-915b-a13350cb0ec8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ee216ff5-e01f-4883-9720-7c00d623582e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:43.054776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:43.056003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:43.056927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:43.056927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:43.056927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:43.056927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:43.056927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 421/860] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:43.057220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6647188-cced-41ac-9548-ca9772e768fb None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.057875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6647188-cced-41ac-9548-ca9772e768fb None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.057875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6647188-cced-41ac-9548-ca9772e768fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.071093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:43.075374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d6647188-cced-41ac-9548-ca9772e768fb None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:43.076546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d6647188-cced-41ac-9548-ca9772e768fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:43.150051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 422/861] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:43.151737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27195b-93fc-4371-915b-a13350cb0ec8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:43.154858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6647188-cced-41ac-9548-ca9772e768fb None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:43.178120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:43.190441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2e102-df05-480f-b306-a5fecefd7c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.191103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2e102-df05-480f-b306-a5fecefd7c82 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.191382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2e102-df05-480f-b306-a5fecefd7c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.191655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2e102-df05-480f-b306-a5fecefd7c82 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.192437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2e102-df05-480f-b306-a5fecefd7c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:43.192743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2e102-df05-480f-b306-a5fecefd7c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.194249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2e102-df05-480f-b306-a5fecefd7c82 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.211769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 440/862] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:43.211769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2e102-df05-480f-b306-a5fecefd7c82 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:43.271587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6647188-cced-41ac-9548-ca9772e768fb None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:43.301674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:43.308304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd78f94-1a1a-4532-aa45-b1789ffa57b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.308974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd78f94-1a1a-4532-aa45-b1789ffa57b9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.309244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd78f94-1a1a-4532-aa45-b1789ffa57b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.309769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd78f94-1a1a-4532-aa45-b1789ffa57b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.310261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd78f94-1a1a-4532-aa45-b1789ffa57b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:43.314950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd78f94-1a1a-4532-aa45-b1789ffa57b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.314950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd78f94-1a1a-4532-aa45-b1789ffa57b9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.319098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2e102-df05-480f-b306-a5fecefd7c82 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:43.329223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 423/863] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:43.329223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd78f94-1a1a-4532-aa45-b1789ffa57b9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:43.365941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d3ad6-f489-4733-b2c7-b69dcbc5a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.366852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d3ad6-f489-4733-b2c7-b69dcbc5a18d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.367197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d3ad6-f489-4733-b2c7-b69dcbc5a18d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.367610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d3ad6-f489-4733-b2c7-b69dcbc5a18d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.368230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d3ad6-f489-4733-b2c7-b69dcbc5a18d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.369586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d3ad6-f489-4733-b2c7-b69dcbc5a18d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-cc5d3ad6-f489-4733-b2c7-b69dcbc5a18d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:43.383449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:43.384623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:43.385485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 441/864] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Jul 13 08:48:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:43.386663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d3ad6-f489-4733-b2c7-b69dcbc5a18d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:43.425002 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:43.448458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd78f94-1a1a-4532-aa45-b1789ffa57b9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:43.485524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d63f1-9e1f-4664-88f8-4a0c14ea406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.486564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d63f1-9e1f-4664-88f8-4a0c14ea406b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.486845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d63f1-9e1f-4664-88f8-4a0c14ea406b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.487243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d63f1-9e1f-4664-88f8-4a0c14ea406b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.488061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d63f1-9e1f-4664-88f8-4a0c14ea406b None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=906fc971a5ee4ffd9503ab13d46cd605) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:43.488370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d63f1-9e1f-4664-88f8-4a0c14ea406b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.489902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d63f1-9e1f-4664-88f8-4a0c14ea406b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.492883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d3ad6-f489-4733-b2c7-b69dcbc5a18d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:43.496374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 424/865] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:43 2022] GET /identity/v3/role_assignments?user.id=906fc971a5ee4ffd9503ab13d46cd605&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:43.497338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d63f1-9e1f-4664-88f8-4a0c14ea406b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:43.540778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6e9aa-16d3-4768-89b8-c20fe332109a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.541642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6e9aa-16d3-4768-89b8-c20fe332109a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.541970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6e9aa-16d3-4768-89b8-c20fe332109a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.542322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6e9aa-16d3-4768-89b8-c20fe332109a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.553829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6e9aa-16d3-4768-89b8-c20fe332109a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.556639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6e9aa-16d3-4768-89b8-c20fe332109a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-16b6e9aa-16d3-4768-89b8-c20fe332109a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:43.575519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:43.576502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:43.577410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:43.577410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:43.577410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:43.577410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:43.577410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:43.577410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:43.577410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:43.577410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:43.582295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 442/866] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:43.583423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6e9aa-16d3-4768-89b8-c20fe332109a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:43.596652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:43.646635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d63f1-9e1f-4664-88f8-4a0c14ea406b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:43.689910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a60ed5-4dd0-41ec-a26c-933c455b5ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.690935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a60ed5-4dd0-41ec-a26c-933c455b5ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.691441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a60ed5-4dd0-41ec-a26c-933c455b5ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.691924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a60ed5-4dd0-41ec-a26c-933c455b5ca7 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.692639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a60ed5-4dd0-41ec-a26c-933c455b5ca7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:43.692950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a60ed5-4dd0-41ec-a26c-933c455b5ca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.695038 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a60ed5-4dd0-41ec-a26c-933c455b5ca7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.699810 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6e9aa-16d3-4768-89b8-c20fe332109a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:43.700565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 425/867] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:48:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:43.705896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a60ed5-4dd0-41ec-a26c-933c455b5ca7 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:43.717370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:43.741930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbcabd-c637-4e58-88a0-0bada065294e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.742784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbcabd-c637-4e58-88a0-0bada065294e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.743079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbcabd-c637-4e58-88a0-0bada065294e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.743356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbcabd-c637-4e58-88a0-0bada065294e None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.744233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbcabd-c637-4e58-88a0-0bada065294e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:43.744697 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbcabd-c637-4e58-88a0-0bada065294e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.746185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbcabd-c637-4e58-88a0-0bada065294e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.760056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 443/868] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:43.768750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbcabd-c637-4e58-88a0-0bada065294e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:43.828585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a60ed5-4dd0-41ec-a26c-933c455b5ca7 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:43.841987 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:43.876698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823502a-5a26-4a55-97d3-82f6aa824cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.877724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbcabd-c637-4e58-88a0-0bada065294e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:43.877874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823502a-5a26-4a55-97d3-82f6aa824cdc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.878346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823502a-5a26-4a55-97d3-82f6aa824cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.878771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823502a-5a26-4a55-97d3-82f6aa824cdc None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.880331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823502a-5a26-4a55-97d3-82f6aa824cdc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:43.880789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823502a-5a26-4a55-97d3-82f6aa824cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.883401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823502a-5a26-4a55-97d3-82f6aa824cdc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.889788 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 426/869] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:48:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:43.922879 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000961ea-fe92-4751-8d27-7a69f6c3e1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:43.923717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000961ea-fe92-4751-8d27-7a69f6c3e1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:43.924115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000961ea-fe92-4751-8d27-7a69f6c3e1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:43.924512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000961ea-fe92-4751-8d27-7a69f6c3e1f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:43.925480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000961ea-fe92-4751-8d27-7a69f6c3e1f9 None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=0b380bc85da64a8bb687e2e5c4577e79) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:43.925901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000961ea-fe92-4751-8d27-7a69f6c3e1f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:43.927577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000961ea-fe92-4751-8d27-7a69f6c3e1f9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:43.934943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 444/870] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:43 2022] GET /identity/v3/role_assignments?user.id=0b380bc85da64a8bb687e2e5c4577e79&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:44.252268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a83a1e-0326-43d3-8be7-acbdbfab222b None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:44.252797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a83a1e-0326-43d3-8be7-acbdbfab222b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:44.252797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a83a1e-0326-43d3-8be7-acbdbfab222b None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:44.254447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 427/871] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:44.262538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a643a-b94f-4283-b4e4-26befc4587b4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:44.262959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a643a-b94f-4283-b4e4-26befc4587b4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:44.263335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a643a-b94f-4283-b4e4-26befc4587b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:44.280932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-144a643a-b94f-4283-b4e4-26befc4587b4 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:44.282178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-144a643a-b94f-4283-b4e4-26befc4587b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:44.355684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 445/872] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:44.379718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a83a1e-0326-43d3-8be7-acbdbfab222b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:44.455466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:44.488939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a83a1e-0326-43d3-8be7-acbdbfab222b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:44.524648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862c81e-0045-4449-aa1b-f0710ca44ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:44.525225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862c81e-0045-4449-aa1b-f0710ca44ed9 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:44.525501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862c81e-0045-4449-aa1b-f0710ca44ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:44.525771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862c81e-0045-4449-aa1b-f0710ca44ed9 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:44.526522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862c81e-0045-4449-aa1b-f0710ca44ed9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:44.527876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862c81e-0045-4449-aa1b-f0710ca44ed9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:44.598122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 428/873] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:48:44 2022] POST /identity/v3/services => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:45.678383 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c20b65-c844-4b64-ac2c-bb88a02a7deb None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:45.679087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c20b65-c844-4b64-ac2c-bb88a02a7deb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:45.679087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c20b65-c844-4b64-ac2c-bb88a02a7deb None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:45.680532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 446/874] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:45.685769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54000b88-26f4-401f-9a37-0bcebea6dac1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:45.686304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54000b88-26f4-401f-9a37-0bcebea6dac1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:45.686460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54000b88-26f4-401f-9a37-0bcebea6dac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:45.704252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-54000b88-26f4-401f-9a37-0bcebea6dac1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:45.705432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-54000b88-26f4-401f-9a37-0bcebea6dac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:45.785580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 429/875] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:45.807465 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c20b65-c844-4b64-ac2c-bb88a02a7deb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:45.873447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:45.923910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c20b65-c844-4b64-ac2c-bb88a02a7deb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:45.960498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c0a94-6049-4ae6-a18c-daa1e206ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:45.961624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c0a94-6049-4ae6-a18c-daa1e206ac82 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:45.962133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c0a94-6049-4ae6-a18c-daa1e206ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:45.962606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c0a94-6049-4ae6-a18c-daa1e206ac82 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:45.963449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c0a94-6049-4ae6-a18c-daa1e206ac82 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:45.965033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c0a94-6049-4ae6-a18c-daa1e206ac82 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-154c0a94-6049-4ae6-a18c-daa1e206ac82 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:45.977509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:45.978554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:45.979431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:45.982361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 447/876] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:48:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:45.990444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54000b88-26f4-401f-9a37-0bcebea6dac1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:46.027760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202b25-7d33-41a4-a95f-024b8a8adb9e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.028091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202b25-7d33-41a4-a95f-024b8a8adb9e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.028326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202b25-7d33-41a4-a95f-024b8a8adb9e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.029929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 448/877] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:46.034909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f729e7-9194-4277-aa0b-bee1e084120a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.035199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f729e7-9194-4277-aa0b-bee1e084120a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.035478 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f729e7-9194-4277-aa0b-bee1e084120a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.047790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:46.052828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e0f729e7-9194-4277-aa0b-bee1e084120a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:46.054057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e0f729e7-9194-4277-aa0b-bee1e084120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:46.102780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54000b88-26f4-401f-9a37-0bcebea6dac1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:46.132451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 449/878] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:46 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:46.133598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59e310-77d3-47f7-bfc9-89df2f4626d5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.133885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59e310-77d3-47f7-bfc9-89df2f4626d5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.134190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59e310-77d3-47f7-bfc9-89df2f4626d5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.135586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 450/879] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:46.140433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152714-acbd-4e99-ae56-328d3bd8fb1c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.140749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152714-acbd-4e99-ae56-328d3bd8fb1c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.140984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152714-acbd-4e99-ae56-328d3bd8fb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.155770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db5489-03f3-40a6-9eef-e1d659ba8db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:46.157575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db5489-03f3-40a6-9eef-e1d659ba8db0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.157908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db5489-03f3-40a6-9eef-e1d659ba8db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.158234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db5489-03f3-40a6-9eef-e1d659ba8db0 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.158573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0d152714-acbd-4e99-ae56-328d3bd8fb1c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:46.158935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db5489-03f3-40a6-9eef-e1d659ba8db0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:46.159300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db5489-03f3-40a6-9eef-e1d659ba8db0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:46.159744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0d152714-acbd-4e99-ae56-328d3bd8fb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:46.160708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db5489-03f3-40a6-9eef-e1d659ba8db0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:46.166842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 430/880] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 13 08:48:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:46.168056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db5489-03f3-40a6-9eef-e1d659ba8db0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:46.215513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:46.232802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 451/881] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:46 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:46.233917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d152714-acbd-4e99-ae56-328d3bd8fb1c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:46.285313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db5489-03f3-40a6-9eef-e1d659ba8db0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:46.307676 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:46.322959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a01aa-a5a1-4d9e-9108-70e51c951261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:46.323726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a01aa-a5a1-4d9e-9108-70e51c951261 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.324004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a01aa-a5a1-4d9e-9108-70e51c951261 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.324278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a01aa-a5a1-4d9e-9108-70e51c951261 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.324989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a01aa-a5a1-4d9e-9108-70e51c951261 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:46.326496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a01aa-a5a1-4d9e-9108-70e51c951261 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:46.348436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d152714-acbd-4e99-ae56-328d3bd8fb1c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:46.381906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e25efc-10ea-4fe0-a036-5559d461ecc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:46.382582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e25efc-10ea-4fe0-a036-5559d461ecc1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.382859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e25efc-10ea-4fe0-a036-5559d461ecc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.383125 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e25efc-10ea-4fe0-a036-5559d461ecc1 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.383822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e25efc-10ea-4fe0-a036-5559d461ecc1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:46.384112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e25efc-10ea-4fe0-a036-5559d461ecc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:46.385459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e25efc-10ea-4fe0-a036-5559d461ecc1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:46.389756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 452/882] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 13 08:48:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:46.390673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e25efc-10ea-4fe0-a036-5559d461ecc1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:46.422414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 431/883] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:48:46 2022] POST /identity/v3/services => generated 269 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:46.464961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:46.533635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e25efc-10ea-4fe0-a036-5559d461ecc1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:46.582521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdbe5e-3698-4930-baf1-017f513516c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:46.583336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdbe5e-3698-4930-baf1-017f513516c4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.583616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdbe5e-3698-4930-baf1-017f513516c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.583889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdbe5e-3698-4930-baf1-017f513516c4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.597625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cdbe5e-3698-4930-baf1-017f513516c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e7cdbe5e-3698-4930-baf1-017f513516c4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:46.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:46.606824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:46.607689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 453/884] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 13 08:48:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:46.614952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a01aa-a5a1-4d9e-9108-70e51c951261 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:46.733505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6783c3-7f9e-4d89-a8aa-553c2af37ae3 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.734079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6783c3-7f9e-4d89-a8aa-553c2af37ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.734079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6783c3-7f9e-4d89-a8aa-553c2af37ae3 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.736275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 454/885] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:46.738842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a01aa-a5a1-4d9e-9108-70e51c951261 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:46.744143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8dbeb-fe38-4753-94a5-ac33f3b46c73 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.744688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8dbeb-fe38-4753-94a5-ac33f3b46c73 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.745184 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8dbeb-fe38-4753-94a5-ac33f3b46c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.764179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-62e8dbeb-fe38-4753-94a5-ac33f3b46c73 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:46.766198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-62e8dbeb-fe38-4753-94a5-ac33f3b46c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:46.775557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc633f-66bd-4e29-aa66-21b24a7b4377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:46.776225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc633f-66bd-4e29-aa66-21b24a7b4377 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.776504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc633f-66bd-4e29-aa66-21b24a7b4377 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.776774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc633f-66bd-4e29-aa66-21b24a7b4377 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.777513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc633f-66bd-4e29-aa66-21b24a7b4377 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:46.777815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc633f-66bd-4e29-aa66-21b24a7b4377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:46.779200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc633f-66bd-4e29-aa66-21b24a7b4377 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:46.788039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 432/886] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 13 08:48:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:46.793333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc633f-66bd-4e29-aa66-21b24a7b4377 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:46.849203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:46.860074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 455/887] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:46.880383 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8dbeb-fe38-4753-94a5-ac33f3b46c73 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:46.918197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:46.920690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc633f-66bd-4e29-aa66-21b24a7b4377 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:46.957973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df8bd4-6a2b-4e6f-8920-1ffafefa91b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:46.959146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df8bd4-6a2b-4e6f-8920-1ffafefa91b0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:46.959646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df8bd4-6a2b-4e6f-8920-1ffafefa91b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:46.960128 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df8bd4-6a2b-4e6f-8920-1ffafefa91b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:46.971850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df8bd4-6a2b-4e6f-8920-1ffafefa91b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:46.973286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df8bd4-6a2b-4e6f-8920-1ffafefa91b0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-52df8bd4-6a2b-4e6f-8920-1ffafefa91b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:46.993254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:46.994300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:46.995169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:46.995169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:46.995169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:46.995169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:46.995169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 433/888] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:46.999994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df8bd4-6a2b-4e6f-8920-1ffafefa91b0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:47.002013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8dbeb-fe38-4753-94a5-ac33f3b46c73 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:47.049692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3a50-d1e7-4dd3-bfba-9f1351652d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:47.049692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3a50-d1e7-4dd3-bfba-9f1351652d9f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:47.049692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3a50-d1e7-4dd3-bfba-9f1351652d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:47.049692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3a50-d1e7-4dd3-bfba-9f1351652d9f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:47.050104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e3a50-d1e7-4dd3-bfba-9f1351652d9f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:47.052252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e3a50-d1e7-4dd3-bfba-9f1351652d9f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4a7e3a50-d1e7-4dd3-bfba-9f1351652d9f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:47.069534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:47.070619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 08:48:47.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:47.076023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 456/889] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:48:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:47.082080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3a50-d1e7-4dd3-bfba-9f1351652d9f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:47.117517 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:47.127774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df8bd4-6a2b-4e6f-8920-1ffafefa91b0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:47.178122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6f8ca-b6d7-409b-8978-41c63362b5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:47.179137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6f8ca-b6d7-409b-8978-41c63362b5ba None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:47.180073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6f8ca-b6d7-409b-8978-41c63362b5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:47.180524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6f8ca-b6d7-409b-8978-41c63362b5ba None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:47.181680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6f8ca-b6d7-409b-8978-41c63362b5ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:47.182458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6f8ca-b6d7-409b-8978-41c63362b5ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:47.184649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6f8ca-b6d7-409b-8978-41c63362b5ba None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:47.190806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:47.194898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3a50-d1e7-4dd3-bfba-9f1351652d9f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:47.201965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 434/890] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:47.207252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6f8ca-b6d7-409b-8978-41c63362b5ba None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:47.237848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4677e-61c8-4cf3-adda-0715c288ad09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:47.238835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4677e-61c8-4cf3-adda-0715c288ad09 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:47.239322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4677e-61c8-4cf3-adda-0715c288ad09 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:47.239802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4677e-61c8-4cf3-adda-0715c288ad09 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:47.240693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4677e-61c8-4cf3-adda-0715c288ad09 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:47.241173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4677e-61c8-4cf3-adda-0715c288ad09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:47.242782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4677e-61c8-4cf3-adda-0715c288ad09 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:47.252821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 457/891] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:48:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:47.254901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4677e-61c8-4cf3-adda-0715c288ad09 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:47.338472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6f8ca-b6d7-409b-8978-41c63362b5ba None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:47.369685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4677e-61c8-4cf3-adda-0715c288ad09 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:47.385728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f90bb-6d91-437e-8ae3-2414b5a04c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:47.386841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f90bb-6d91-437e-8ae3-2414b5a04c45 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:47.387138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f90bb-6d91-437e-8ae3-2414b5a04c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:47.387407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f90bb-6d91-437e-8ae3-2414b5a04c45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:47.397740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f90bb-6d91-437e-8ae3-2414b5a04c45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:47.399976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f90bb-6d91-437e-8ae3-2414b5a04c45 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:47.407958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a950d4-108a-4142-93fb-a8ae4aa9aac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:47.408925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a950d4-108a-4142-93fb-a8ae4aa9aac0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:47.409405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a950d4-108a-4142-93fb-a8ae4aa9aac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:47.409876 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a950d4-108a-4142-93fb-a8ae4aa9aac0 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:47.410795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a950d4-108a-4142-93fb-a8ae4aa9aac0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:47.411311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a950d4-108a-4142-93fb-a8ae4aa9aac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:47.412936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a950d4-108a-4142-93fb-a8ae4aa9aac0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-cc3f90bb-6d91-437e-8ae3-2414b5a04c45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:47.417311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:47.418274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:47.419203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:47.419203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:47.419203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:47.419203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:47.419203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:47.419203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:47.419203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:47.419203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:47.419203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:47.419203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:47.427130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 435/892] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 13 08:48:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:47.427130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f90bb-6d91-437e-8ae3-2414b5a04c45 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:47.427335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 458/893] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:48:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:47.428618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a950d4-108a-4142-93fb-a8ae4aa9aac0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:47.447111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:47.461258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:47.549566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a950d4-108a-4142-93fb-a8ae4aa9aac0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:47.555223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f90bb-6d91-437e-8ae3-2414b5a04c45 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:47.587165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa41a5-bfac-4461-8825-8d55c0cf3fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:47.588166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa41a5-bfac-4461-8825-8d55c0cf3fbf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:47.588555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa41a5-bfac-4461-8825-8d55c0cf3fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:47.588779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa41a5-bfac-4461-8825-8d55c0cf3fbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:47.590219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa41a5-bfac-4461-8825-8d55c0cf3fbf None admin] RBAC: Adding query filter params (interface=public, service_id=f023632a7ff74db2b8e054f4fd8be5e3, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:47.590799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa41a5-bfac-4461-8825-8d55c0cf3fbf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:47.592251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa41a5-bfac-4461-8825-8d55c0cf3fbf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:47.596381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fce91-d504-49c9-b7ae-a2fabf3f039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:47.597427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fce91-d504-49c9-b7ae-a2fabf3f039c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:47.597954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fce91-d504-49c9-b7ae-a2fabf3f039c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:47.598482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fce91-d504-49c9-b7ae-a2fabf3f039c None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:47.602073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 459/894] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 13 08:48:47 2022] GET /identity/v3/endpoints?service_id=f023632a7ff74db2b8e054f4fd8be5e3&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:47.604147 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fce91-d504-49c9-b7ae-a2fabf3f039c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:47.604719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fce91-d504-49c9-b7ae-a2fabf3f039c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:47.606344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fce91-d504-49c9-b7ae-a2fabf3f039c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:47.611293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa41a5-bfac-4461-8825-8d55c0cf3fbf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:47.632496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 436/895] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 13 08:48:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:47.633542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fce91-d504-49c9-b7ae-a2fabf3f039c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:47.723809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:47.745209 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:47.755040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fce91-d504-49c9-b7ae-a2fabf3f039c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:47.758470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa41a5-bfac-4461-8825-8d55c0cf3fbf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:47.793014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44dde1-98fe-4d6a-987c-0359a86da78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:47.794201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44dde1-98fe-4d6a-987c-0359a86da78e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:47.795227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44dde1-98fe-4d6a-987c-0359a86da78e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:47.795965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44dde1-98fe-4d6a-987c-0359a86da78e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:47.801292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d74c3-8169-4cde-a770-2b6560bfcb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:47.802205 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d74c3-8169-4cde-a770-2b6560bfcb44 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:47.802565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d74c3-8169-4cde-a770-2b6560bfcb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:47.802913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d74c3-8169-4cde-a770-2b6560bfcb44 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:47.803664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d74c3-8169-4cde-a770-2b6560bfcb44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:47.805111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d74c3-8169-4cde-a770-2b6560bfcb44 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:47.818072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 460/896] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 13 08:48:47 2022] GET /identity/v3/services => generated 1268 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 08:48:47.875195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44dde1-98fe-4d6a-987c-0359a86da78e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:47.876948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44dde1-98fe-4d6a-987c-0359a86da78e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:47.954526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:47.988001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 437/897] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:47 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:48:47.993447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d74c3-8169-4cde-a770-2b6560bfcb44 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:48.101234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d74c3-8169-4cde-a770-2b6560bfcb44 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:48.136295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76d16f-da41-4be9-aa9e-09b743d1736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:48.137015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76d16f-da41-4be9-aa9e-09b743d1736c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:48.137295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76d16f-da41-4be9-aa9e-09b743d1736c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:48.137570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76d16f-da41-4be9-aa9e-09b743d1736c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:48.149400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76d16f-da41-4be9-aa9e-09b743d1736c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:48.150893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76d16f-da41-4be9-aa9e-09b743d1736c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5c76d16f-da41-4be9-aa9e-09b743d1736c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:48.169064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:48.170363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:48.171679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:48.171679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:48.171679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:48.171679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:48.171679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 461/898] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:48.176427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44dde1-98fe-4d6a-987c-0359a86da78e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:48.287311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44dde1-98fe-4d6a-987c-0359a86da78e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:48.324885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf0e8f-fc63-41f2-8b8c-7cdb41185258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:48.326063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf0e8f-fc63-41f2-8b8c-7cdb41185258 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:48.326549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf0e8f-fc63-41f2-8b8c-7cdb41185258 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:48.327017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf0e8f-fc63-41f2-8b8c-7cdb41185258 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:48.328013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf0e8f-fc63-41f2-8b8c-7cdb41185258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:48.329292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf0e8f-fc63-41f2-8b8c-7cdb41185258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:48.330884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf0e8f-fc63-41f2-8b8c-7cdb41185258 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:48.352523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 438/899] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:48.358274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76d16f-da41-4be9-aa9e-09b743d1736c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:48.369661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:48.465900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76d16f-da41-4be9-aa9e-09b743d1736c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:48.501079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf72c3-2c92-427d-9e2f-69c83822bf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:48.502374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf72c3-2c92-427d-9e2f-69c83822bf90 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:48.502881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf72c3-2c92-427d-9e2f-69c83822bf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:48.503434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf72c3-2c92-427d-9e2f-69c83822bf90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:48.511038 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf72c3-2c92-427d-9e2f-69c83822bf90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:48.512712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf72c3-2c92-427d-9e2f-69c83822bf90 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-43bf72c3-2c92-427d-9e2f-69c83822bf90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:48.528488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:48.529411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:48.530348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:48.530348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:48.530348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:48.530348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:48.530348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:48.530348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:48.530348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:48.530348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:48.532907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 462/900] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:48.538388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf72c3-2c92-427d-9e2f-69c83822bf90 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:48.539198 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17c144-8464-4a58-8030-5514e7eb9248 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:48.539707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17c144-8464-4a58-8030-5514e7eb9248 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:48.540180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17c144-8464-4a58-8030-5514e7eb9248 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:48.543487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 439/901] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:48.545424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67c208-daaa-4628-ad84-203f4e65e26e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:48.545708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67c208-daaa-4628-ad84-203f4e65e26e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:48.545983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67c208-daaa-4628-ad84-203f4e65e26e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:48.548139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 440/902] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:48.549236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bba09-15c4-478a-baa4-d02ed8ada4a1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:48.549537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bba09-15c4-478a-baa4-d02ed8ada4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:48.549818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bba09-15c4-478a-baa4-d02ed8ada4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:48.569121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0e4bba09-15c4-478a-baa4-d02ed8ada4a1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:48.570338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0e4bba09-15c4-478a-baa4-d02ed8ada4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:48.598320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:48.615172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:48.646248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 441/903] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:48.646248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a325f7-7465-4d79-a753-904294145dbe None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:48.646248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a325f7-7465-4d79-a753-904294145dbe None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:48.646248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a325f7-7465-4d79-a753-904294145dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:48.658367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf72c3-2c92-427d-9e2f-69c83822bf90 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:48.665097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d0a325f7-7465-4d79-a753-904294145dbe None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:48.666615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d0a325f7-7465-4d79-a753-904294145dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:48.695650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6f1a8-9878-4092-aa1a-6e5b0c9ac4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:48.696302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6f1a8-9878-4092-aa1a-6e5b0c9ac4ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:48.696580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6f1a8-9878-4092-aa1a-6e5b0c9ac4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:48.696852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6f1a8-9878-4092-aa1a-6e5b0c9ac4ec None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:48.697633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6f1a8-9878-4092-aa1a-6e5b0c9ac4ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:48.697936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6f1a8-9878-4092-aa1a-6e5b0c9ac4ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:48.699403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6f1a8-9878-4092-aa1a-6e5b0c9ac4ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:48.711561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 463/904] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:48.712506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6f1a8-9878-4092-aa1a-6e5b0c9ac4ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:48.740151 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 442/905] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:48.741259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a325f7-7465-4d79-a753-904294145dbe None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:48.838403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6f1a8-9878-4092-aa1a-6e5b0c9ac4ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:48.838967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:48.847792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a325f7-7465-4d79-a753-904294145dbe None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:48.866321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:48.873956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08610001-3933-450e-9923-3d4bedd6fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:48.874800 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08610001-3933-450e-9923-3d4bedd6fc73 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:48.875081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08610001-3933-450e-9923-3d4bedd6fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:48.876055 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08610001-3933-450e-9923-3d4bedd6fc73 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:48.876719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08610001-3933-450e-9923-3d4bedd6fc73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:48.878161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08610001-3933-450e-9923-3d4bedd6fc73 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:48.884706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b075f9-c37a-49bb-b85d-ec0e3d30157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:48.885359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b075f9-c37a-49bb-b85d-ec0e3d30157b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:48.885645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b075f9-c37a-49bb-b85d-ec0e3d30157b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:48.885921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b075f9-c37a-49bb-b85d-ec0e3d30157b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:48.886775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b075f9-c37a-49bb-b85d-ec0e3d30157b None admin] RBAC: Adding query filter params (role.id=609bba755f3f46f1b91a203b0a8ea404, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=0b380bc85da64a8bb687e2e5c4577e79) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:48.887089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b075f9-c37a-49bb-b85d-ec0e3d30157b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:48.888494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b075f9-c37a-49bb-b85d-ec0e3d30157b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-08610001-3933-450e-9923-3d4bedd6fc73 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:48.890858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:48.892156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:48.892995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 464/906] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Jul 13 08:48:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:48.894772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08610001-3933-450e-9923-3d4bedd6fc73 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:48.911051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 443/907] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:48 2022] GET /identity/v3/role_assignments?user.id=0b380bc85da64a8bb687e2e5c4577e79&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=609bba755f3f46f1b91a203b0a8ea404 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:48.911051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b075f9-c37a-49bb-b85d-ec0e3d30157b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:49.033370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08610001-3933-450e-9923-3d4bedd6fc73 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:49.042258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b075f9-c37a-49bb-b85d-ec0e3d30157b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:49.076665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a3d25-80d5-40af-9cbf-cbaadbf0777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:49.077335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3d25-80d5-40af-9cbf-cbaadbf0777e None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:49.081385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3d25-80d5-40af-9cbf-cbaadbf0777e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:49.081385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3d25-80d5-40af-9cbf-cbaadbf0777e None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:49.081385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a3d25-80d5-40af-9cbf-cbaadbf0777e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:49.081385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a3d25-80d5-40af-9cbf-cbaadbf0777e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:49.091096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc43411-f7b5-4fcd-a52b-6d5736db7037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:49.091759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc43411-f7b5-4fcd-a52b-6d5736db7037 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:49.092038 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc43411-f7b5-4fcd-a52b-6d5736db7037 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:49.092306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc43411-f7b5-4fcd-a52b-6d5736db7037 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:49.093013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc43411-f7b5-4fcd-a52b-6d5736db7037 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:49.093314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc43411-f7b5-4fcd-a52b-6d5736db7037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:49.095358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc43411-f7b5-4fcd-a52b-6d5736db7037 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:49.100237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 444/908] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:48:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:49.106147 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc43411-f7b5-4fcd-a52b-6d5736db7037 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:49.129731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:49.181962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 465/909] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:48:48 2022] POST /identity/v3/services => generated 241 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:49.228024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc43411-f7b5-4fcd-a52b-6d5736db7037 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:49.275391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa3efc-1674-4736-bce6-b265c0fdce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:49.276453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa3efc-1674-4736-bce6-b265c0fdce1f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:49.276986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa3efc-1674-4736-bce6-b265c0fdce1f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:49.277490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa3efc-1674-4736-bce6-b265c0fdce1f None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:49.278435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa3efc-1674-4736-bce6-b265c0fdce1f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:49.279027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa3efc-1674-4736-bce6-b265c0fdce1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:49.280692 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa3efc-1674-4736-bce6-b265c0fdce1f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:49.292065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 445/910] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:48:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:49.293816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a3d25-80d5-40af-9cbf-cbaadbf0777e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:49.359263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:49.418407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a3d25-80d5-40af-9cbf-cbaadbf0777e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:49.455239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afc830-a49e-48bc-9fd6-dbfc1c0e4f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:49.455904 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afc830-a49e-48bc-9fd6-dbfc1c0e4f95 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:49.456180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afc830-a49e-48bc-9fd6-dbfc1c0e4f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:49.456455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afc830-a49e-48bc-9fd6-dbfc1c0e4f95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:49.457247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69afc830-a49e-48bc-9fd6-dbfc1c0e4f95 None admin] RBAC: Adding query filter params (interface=public, service_id=775e06c522c047c9baf8bbcbeff9ffc8, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:49.457544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69afc830-a49e-48bc-9fd6-dbfc1c0e4f95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:49.458977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69afc830-a49e-48bc-9fd6-dbfc1c0e4f95 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:49.464570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 466/911] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 13 08:48:49 2022] GET /identity/v3/endpoints?service_id=775e06c522c047c9baf8bbcbeff9ffc8&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:49.470843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa3efc-1674-4736-bce6-b265c0fdce1f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:49.556133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:49.576609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa3efc-1674-4736-bce6-b265c0fdce1f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:49.618387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4923c4-c766-4da3-b92f-860342b8e9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:49.619053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4923c4-c766-4da3-b92f-860342b8e9dd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:49.619333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4923c4-c766-4da3-b92f-860342b8e9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:49.619605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4923c4-c766-4da3-b92f-860342b8e9dd None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:49.620281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4923c4-c766-4da3-b92f-860342b8e9dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:49.621636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4923c4-c766-4da3-b92f-860342b8e9dd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:49.629031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 446/912] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 13 08:48:49 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 08:48:50.001719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908af818-bd26-4115-ac05-c402d6d17162 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:50.002335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908af818-bd26-4115-ac05-c402d6d17162 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:50.002335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908af818-bd26-4115-ac05-c402d6d17162 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:50.003881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 467/913] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:50.009086 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb9c7a-a1d9-49a2-908b-02387871fce6 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:50.009489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb9c7a-a1d9-49a2-908b-02387871fce6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:50.009703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb9c7a-a1d9-49a2-908b-02387871fce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:50.027065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-66eb9c7a-a1d9-49a2-908b-02387871fce6 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:50.028251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-66eb9c7a-a1d9-49a2-908b-02387871fce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:50.106628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 447/914] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:50.130204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908af818-bd26-4115-ac05-c402d6d17162 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:50.239995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908af818-bd26-4115-ac05-c402d6d17162 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:50.257634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:50.275608 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e1023-a215-4f8a-b46a-3769d4296b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:50.279382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e1023-a215-4f8a-b46a-3769d4296b57 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:50.279382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e1023-a215-4f8a-b46a-3769d4296b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:50.279382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e1023-a215-4f8a-b46a-3769d4296b57 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:50.279382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e1023-a215-4f8a-b46a-3769d4296b57 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:50.279382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e1023-a215-4f8a-b46a-3769d4296b57 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-185e1023-a215-4f8a-b46a-3769d4296b57 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:50.291863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:50.292857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:50.293688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 468/915] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:48:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:50.299764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb9c7a-a1d9-49a2-908b-02387871fce6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:50.383062 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:50.410314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb9c7a-a1d9-49a2-908b-02387871fce6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:50.445799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1c577-811a-4791-8d3e-a8a60174f9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:50.446489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1c577-811a-4791-8d3e-a8a60174f9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:50.446770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1c577-811a-4791-8d3e-a8a60174f9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:50.447039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1c577-811a-4791-8d3e-a8a60174f9d8 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:50.447732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1c577-811a-4791-8d3e-a8a60174f9d8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:50.448034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1c577-811a-4791-8d3e-a8a60174f9d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:50.449383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1c577-811a-4791-8d3e-a8a60174f9d8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:50.455372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 448/916] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:48:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:50.460617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e1023-a215-4f8a-b46a-3769d4296b57 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:50.568291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e1023-a215-4f8a-b46a-3769d4296b57 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:50.603374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5db392-f0e7-4ebd-a96c-af1a7841fddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:50.604033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db392-f0e7-4ebd-a96c-af1a7841fddb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:50.604316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db392-f0e7-4ebd-a96c-af1a7841fddb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:50.604592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db392-f0e7-4ebd-a96c-af1a7841fddb None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:50.605295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5db392-f0e7-4ebd-a96c-af1a7841fddb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:50.605600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5db392-f0e7-4ebd-a96c-af1a7841fddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:50.607007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5db392-f0e7-4ebd-a96c-af1a7841fddb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:50.612729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 469/917] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 13 08:48:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:50.618296 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1c577-811a-4791-8d3e-a8a60174f9d8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:50.724852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1c577-811a-4791-8d3e-a8a60174f9d8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:50.759710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040c15a-edc9-4d88-864d-a1b30815661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:50.760367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040c15a-edc9-4d88-864d-a1b30815661e None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:50.760646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040c15a-edc9-4d88-864d-a1b30815661e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:50.760918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040c15a-edc9-4d88-864d-a1b30815661e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:50.761661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040c15a-edc9-4d88-864d-a1b30815661e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:50.763045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040c15a-edc9-4d88-864d-a1b30815661e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:50.774349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:50.863352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 449/918] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 13 08:48:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:48:51.074506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3b7b-c5b5-4df0-ad35-c67b76400348 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.075504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3b7b-c5b5-4df0-ad35-c67b76400348 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.076020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c3b7b-c5b5-4df0-ad35-c67b76400348 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.077840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 470/919] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:51.084014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf24a0-c31a-4274-b3b1-6fc349e37cb8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.084448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf24a0-c31a-4274-b3b1-6fc349e37cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.084613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf24a0-c31a-4274-b3b1-6fc349e37cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.102353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-53cf24a0-c31a-4274-b3b1-6fc349e37cb8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:51.103532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-53cf24a0-c31a-4274-b3b1-6fc349e37cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:51.178300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 450/920] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:51.200907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c3b7b-c5b5-4df0-ad35-c67b76400348 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:51.258194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:51.322576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c3b7b-c5b5-4df0-ad35-c67b76400348 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:51.358909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35e08f-a4f6-4555-834d-8548665d2a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:51.359758 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35e08f-a4f6-4555-834d-8548665d2a13 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.360676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35e08f-a4f6-4555-834d-8548665d2a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.360960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35e08f-a4f6-4555-834d-8548665d2a13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.372475 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35e08f-a4f6-4555-834d-8548665d2a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-fc35e08f-a4f6-4555-834d-8548665d2a13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:51.381659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:51.382652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:51.383501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 471/921] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 13 08:48:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:51.390773 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf24a0-c31a-4274-b3b1-6fc349e37cb8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:51.392083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86f33c-0b89-4f3c-8381-0555e3f27550 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.392423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86f33c-0b89-4f3c-8381-0555e3f27550 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.392706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86f33c-0b89-4f3c-8381-0555e3f27550 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.394306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 472/922] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:51.399371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91565f9f-6ec1-4bb1-ac09-1214bfd7abe0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.399654 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91565f9f-6ec1-4bb1-ac09-1214bfd7abe0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.399932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91565f9f-6ec1-4bb1-ac09-1214bfd7abe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.418252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-91565f9f-6ec1-4bb1-ac09-1214bfd7abe0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:51.419700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-91565f9f-6ec1-4bb1-ac09-1214bfd7abe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:51.494861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 473/923] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:51.507261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf24a0-c31a-4274-b3b1-6fc349e37cb8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:51.521121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91565f9f-6ec1-4bb1-ac09-1214bfd7abe0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:51.533841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:51.555932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41b51a-a31c-4906-834e-af4c746b2058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:51.557082 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41b51a-a31c-4906-834e-af4c746b2058 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.562175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41b51a-a31c-4906-834e-af4c746b2058 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.562175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41b51a-a31c-4906-834e-af4c746b2058 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.562175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41b51a-a31c-4906-834e-af4c746b2058 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:51.562175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41b51a-a31c-4906-834e-af4c746b2058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:51.562175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41b51a-a31c-4906-834e-af4c746b2058 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:51.562575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:51.574468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 451/924] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 13 08:48:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:51.581612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41b51a-a31c-4906-834e-af4c746b2058 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:51.635445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91565f9f-6ec1-4bb1-ac09-1214bfd7abe0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:51.673299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab52a5-aaf8-4ff2-b92f-644505062865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:51.674183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab52a5-aaf8-4ff2-b92f-644505062865 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.674565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab52a5-aaf8-4ff2-b92f-644505062865 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.674933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab52a5-aaf8-4ff2-b92f-644505062865 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.675680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab52a5-aaf8-4ff2-b92f-644505062865 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:51.677280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab52a5-aaf8-4ff2-b92f-644505062865 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1fab52a5-aaf8-4ff2-b92f-644505062865 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:51.690319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:51.692003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:51.694051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:51.695761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41b51a-a31c-4906-834e-af4c746b2058 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:51.697395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 474/925] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:48:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:51.702249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab52a5-aaf8-4ff2-b92f-644505062865 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:51.730887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5b87c-b22b-40d2-9d34-35e889aea807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:51.731681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5b87c-b22b-40d2-9d34-35e889aea807 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.731966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5b87c-b22b-40d2-9d34-35e889aea807 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.732244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5b87c-b22b-40d2-9d34-35e889aea807 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.745458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5b87c-b22b-40d2-9d34-35e889aea807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:51.747092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5b87c-b22b-40d2-9d34-35e889aea807 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-52b5b87c-b22b-40d2-9d34-35e889aea807 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:51.766851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:51.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:51.770506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:51.770506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:51.770506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:51.770506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:51.770506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 452/926] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:51.773900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5b87c-b22b-40d2-9d34-35e889aea807 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:51.799881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:51.803497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab52a5-aaf8-4ff2-b92f-644505062865 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:51.813763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:51.838171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89499a-465b-4de2-8a99-c8c12e6b4458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:51.838831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89499a-465b-4de2-8a99-c8c12e6b4458 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.839165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89499a-465b-4de2-8a99-c8c12e6b4458 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.839445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89499a-465b-4de2-8a99-c8c12e6b4458 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.840219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89499a-465b-4de2-8a99-c8c12e6b4458 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:51.840452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89499a-465b-4de2-8a99-c8c12e6b4458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:51.841937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89499a-465b-4de2-8a99-c8c12e6b4458 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:51.851695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 475/927] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 13 08:48:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:51.852639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89499a-465b-4de2-8a99-c8c12e6b4458 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:51.893492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5b87c-b22b-40d2-9d34-35e889aea807 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:51.932511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9079b-e16e-4389-93a8-9bb545a15405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:51.933473 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9079b-e16e-4389-93a8-9bb545a15405 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.934047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9079b-e16e-4389-93a8-9bb545a15405 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.934330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9079b-e16e-4389-93a8-9bb545a15405 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.935045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9079b-e16e-4389-93a8-9bb545a15405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:51.935474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9079b-e16e-4389-93a8-9bb545a15405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:51.936830 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9079b-e16e-4389-93a8-9bb545a15405 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:51.954010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 453/928] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:51.955174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04654f-dab5-4f95-8174-014d06e0fb59 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.955463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04654f-dab5-4f95-8174-014d06e0fb59 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.955738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04654f-dab5-4f95-8174-014d06e0fb59 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.960472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89499a-465b-4de2-8a99-c8c12e6b4458 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:51.960620 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 454/929] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:51.960620 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04654f-dab5-4f95-8174-014d06e0fb59 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:51.995611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a356c4-aafd-4d42-8745-4131ad0cdca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:51.996285 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a356c4-aafd-4d42-8745-4131ad0cdca8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:51.996568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a356c4-aafd-4d42-8745-4131ad0cdca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:51.996845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a356c4-aafd-4d42-8745-4131ad0cdca8 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:51.997559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a356c4-aafd-4d42-8745-4131ad0cdca8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:51.997865 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a356c4-aafd-4d42-8745-4131ad0cdca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:51.999315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a356c4-aafd-4d42-8745-4131ad0cdca8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:52.009888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 476/930] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 13 08:48:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:52.009888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab316b5-3e6b-4fe3-be10-5feeedef2128 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.009888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab316b5-3e6b-4fe3-be10-5feeedef2128 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.009888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab316b5-3e6b-4fe3-be10-5feeedef2128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.024841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2ab316b5-3e6b-4fe3-be10-5feeedef2128 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:52.026016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2ab316b5-3e6b-4fe3-be10-5feeedef2128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:52.048888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:52.069908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:52.077931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04654f-dab5-4f95-8174-014d06e0fb59 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:52.098080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 477/931] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:52.098080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab316b5-3e6b-4fe3-be10-5feeedef2128 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:52.119956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f141c79-9432-4cf9-b8a4-ccb84523bda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:52.120795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f141c79-9432-4cf9-b8a4-ccb84523bda6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.121071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f141c79-9432-4cf9-b8a4-ccb84523bda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.121346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f141c79-9432-4cf9-b8a4-ccb84523bda6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.129182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f141c79-9432-4cf9-b8a4-ccb84523bda6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:52.130775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f141c79-9432-4cf9-b8a4-ccb84523bda6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3f141c79-9432-4cf9-b8a4-ccb84523bda6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:52.148570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:52.149550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 455/932] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 13 08:48:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:52.150603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f141c79-9432-4cf9-b8a4-ccb84523bda6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:52.206109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab316b5-3e6b-4fe3-be10-5feeedef2128 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:52.242195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d7840-447a-475c-830b-c64d5cfdb5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:52.243009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d7840-447a-475c-830b-c64d5cfdb5ba None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.243129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d7840-447a-475c-830b-c64d5cfdb5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.243382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d7840-447a-475c-830b-c64d5cfdb5ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.244223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d7840-447a-475c-830b-c64d5cfdb5ba None admin] RBAC: Adding query filter params (interface=public, service_id=9355914168f140eba5a39cacfed3333b, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:52.244472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d7840-447a-475c-830b-c64d5cfdb5ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:52.245887 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d7840-447a-475c-830b-c64d5cfdb5ba None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:52.255875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 478/933] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 13 08:48:52 2022] GET /identity/v3/endpoints?service_id=9355914168f140eba5a39cacfed3333b&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:52.255875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d7840-447a-475c-830b-c64d5cfdb5ba None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:52.270269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f141c79-9432-4cf9-b8a4-ccb84523bda6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:52.278865 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:52.308406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf94bf-2c17-4366-8567-c8edf846a3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:52.309223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf94bf-2c17-4366-8567-c8edf846a3da None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.309481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf94bf-2c17-4366-8567-c8edf846a3da None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.309756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf94bf-2c17-4366-8567-c8edf846a3da None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.310441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf94bf-2c17-4366-8567-c8edf846a3da None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:52.311924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf94bf-2c17-4366-8567-c8edf846a3da None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-0bdf94bf-2c17-4366-8567-c8edf846a3da None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:52.324763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:52.325782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 08:48:52.326827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:52.331113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 456/934] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Jul 13 08:48:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:52.331877 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf94bf-2c17-4366-8567-c8edf846a3da None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:52.357382 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:52.366989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d7840-447a-475c-830b-c64d5cfdb5ba None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:52.401458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993253a0-d24c-4622-87d9-2cba8d349790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:52.402155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993253a0-d24c-4622-87d9-2cba8d349790 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.402440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993253a0-d24c-4622-87d9-2cba8d349790 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.402721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993253a0-d24c-4622-87d9-2cba8d349790 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.403557 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993253a0-d24c-4622-87d9-2cba8d349790 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:52.403962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993253a0-d24c-4622-87d9-2cba8d349790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:52.405408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993253a0-d24c-4622-87d9-2cba8d349790 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:52.423837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 479/935] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 13 08:48:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:52.424646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993253a0-d24c-4622-87d9-2cba8d349790 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:52.450772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf94bf-2c17-4366-8567-c8edf846a3da None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:52.488819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5847b-a817-4572-a485-da93dfa5b519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:52.489767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5847b-a817-4572-a485-da93dfa5b519 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.490292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5847b-a817-4572-a485-da93dfa5b519 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.490864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5847b-a817-4572-a485-da93dfa5b519 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.491856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5847b-a817-4572-a485-da93dfa5b519 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:52.493220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5847b-a817-4572-a485-da93dfa5b519 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:52.499453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 457/936] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 13 08:48:52 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 08:48:52.500842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5847b-a817-4572-a485-da93dfa5b519 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:52.515291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:52.540949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993253a0-d24c-4622-87d9-2cba8d349790 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:52.591375 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb7353-3b0e-4d62-a37d-d9727100e1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:52.593158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb7353-3b0e-4d62-a37d-d9727100e1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.593665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb7353-3b0e-4d62-a37d-d9727100e1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.594382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb7353-3b0e-4d62-a37d-d9727100e1f2 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.595355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb7353-3b0e-4d62-a37d-d9727100e1f2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:52.596144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb7353-3b0e-4d62-a37d-d9727100e1f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:52.598662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb7353-3b0e-4d62-a37d-d9727100e1f2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:52.609132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 480/937] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:48:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:52.615070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb7353-3b0e-4d62-a37d-d9727100e1f2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:52.623515 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:52.646269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5847b-a817-4572-a485-da93dfa5b519 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:52.692303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ad147-a826-4d07-be17-90b4afb4116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:52.693101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ad147-a826-4d07-be17-90b4afb4116b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.693387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ad147-a826-4d07-be17-90b4afb4116b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.693665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ad147-a826-4d07-be17-90b4afb4116b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.710089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ad147-a826-4d07-be17-90b4afb4116b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:52.710739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ad147-a826-4d07-be17-90b4afb4116b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:52.733702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb7353-3b0e-4d62-a37d-d9727100e1f2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-154ad147-a826-4d07-be17-90b4afb4116b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:52.736188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:52.737070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:52.738567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:52.738567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:52.738567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:52.738567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:52.738567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 458/938] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:52.743002 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ad147-a826-4d07-be17-90b4afb4116b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:52.776647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c36335-4ef9-4263-b5de-f837fad07c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:52.777526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c36335-4ef9-4263-b5de-f837fad07c04 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.777946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c36335-4ef9-4263-b5de-f837fad07c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.778360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c36335-4ef9-4263-b5de-f837fad07c04 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.779071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c36335-4ef9-4263-b5de-f837fad07c04 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:52.779381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c36335-4ef9-4263-b5de-f837fad07c04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:52.781844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c36335-4ef9-4263-b5de-f837fad07c04 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:52.787786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 481/939] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:48:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:52.793158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c36335-4ef9-4263-b5de-f837fad07c04 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:52.802290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:52.856929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ad147-a826-4d07-be17-90b4afb4116b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:52.896262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f7057-9818-4564-aab3-86443ab5f9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:52.897033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f7057-9818-4564-aab3-86443ab5f9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.897320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f7057-9818-4564-aab3-86443ab5f9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.897607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f7057-9818-4564-aab3-86443ab5f9b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.898386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f7057-9818-4564-aab3-86443ab5f9b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:52.898697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f7057-9818-4564-aab3-86443ab5f9b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:52.899390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c36335-4ef9-4263-b5de-f837fad07c04 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:52.900072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f7057-9818-4564-aab3-86443ab5f9b0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:52.901775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:52.918274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 459/940] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:52.919748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f7057-9818-4564-aab3-86443ab5f9b0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:52.935597 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159bb56-44f8-4ace-9a61-991754f974c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:52.936250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159bb56-44f8-4ace-9a61-991754f974c0 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:52.936526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159bb56-44f8-4ace-9a61-991754f974c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:52.936794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159bb56-44f8-4ace-9a61-991754f974c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:52.937554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159bb56-44f8-4ace-9a61-991754f974c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:52.939926 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159bb56-44f8-4ace-9a61-991754f974c0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:53.031483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f7057-9818-4564-aab3-86443ab5f9b0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:53.036728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 482/941] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 13 08:48:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:48:53.038211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986cafe-fba0-4f59-a47d-401e3550ac61 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:53.038504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986cafe-fba0-4f59-a47d-401e3550ac61 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:53.038780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986cafe-fba0-4f59-a47d-401e3550ac61 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:53.042076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 483/942] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:53.046086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ca06-4566-41f6-a8fd-fc741a6df697 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:53.046433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ca06-4566-41f6-a8fd-fc741a6df697 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:53.046719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ca06-4566-41f6-a8fd-fc741a6df697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:53.069977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8e31ca06-4566-41f6-a8fd-fc741a6df697 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:53.071561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8e31ca06-4566-41f6-a8fd-fc741a6df697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:53.084757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a2574-3af2-4846-afdb-b1d1592c4ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:53.085903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a2574-3af2-4846-afdb-b1d1592c4ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:53.086363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a2574-3af2-4846-afdb-b1d1592c4ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:53.086821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a2574-3af2-4846-afdb-b1d1592c4ef3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:53.095095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a2574-3af2-4846-afdb-b1d1592c4ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:53.096699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a2574-3af2-4846-afdb-b1d1592c4ef3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ab3a2574-3af2-4846-afdb-b1d1592c4ef3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:53.114139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:53.115159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:53.116065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:53.116065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:53.116065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:53.116065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:53.116065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:53.116065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:53.116065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:53.116065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:53.120441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 460/943] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:53.121417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a2574-3af2-4846-afdb-b1d1592c4ef3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:53.162736 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:53.186486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 484/944] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:53.203545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:53.207764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ca06-4566-41f6-a8fd-fc741a6df697 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:53.255373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a2574-3af2-4846-afdb-b1d1592c4ef3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:53.311433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9ccea-dea2-48b1-8746-4e357f1259cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:53.313556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9ccea-dea2-48b1-8746-4e357f1259cc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:53.314382 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9ccea-dea2-48b1-8746-4e357f1259cc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:53.315251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9ccea-dea2-48b1-8746-4e357f1259cc None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:53.316980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9ccea-dea2-48b1-8746-4e357f1259cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:53.317825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9ccea-dea2-48b1-8746-4e357f1259cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:53.320795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9ccea-dea2-48b1-8746-4e357f1259cc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:53.333069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ca06-4566-41f6-a8fd-fc741a6df697 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:53.354301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 461/945] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:53.360793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9ccea-dea2-48b1-8746-4e357f1259cc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:53.379558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:53.380275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:53.380556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:53.380830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:53.381583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:53.383154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:53.384583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WARNING keystone.server.flask.common [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 13 08:48:53.427190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.sql.core [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 08:48:53.427190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 13 08:48:53.427190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [parameters: {'id': '64d955c009de4895adf80ab54a7464ff', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 13 08:48:53.427190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80287) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64d955c009de4895adf80ab54a7464ff.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64d955c009de4895adf80ab54a7464ff. Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application conn.query(q) Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 08:48:53.449947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result.read() Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 13 08:48:53.450864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application next(self.gen) Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application yield resource Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application next(self.gen) Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise self.value Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application session.commit() Jul 13 08:48:53.451819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.session.flush() Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application compat.raise_( Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 08:48:53.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 08:48:53.459901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application conn.query(q) Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result.read() Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 08:48:53.461852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [parameters: {'id': '64d955c009de4895adf80ab54a7464ff', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:53.463944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:53.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 64d955c009de4895adf80ab54a7464ff. Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 485/946] 10.222.0.40 () {60 vars in 1140 bytes} [Wed Jul 13 08:48:53 2022] POST /identity/v3/projects => generated 262 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:53.467966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:53.496160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9ccea-dea2-48b1-8746-4e357f1259cc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:53.509740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:53.519316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:53.539308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628306-2b4c-4eec-9ae8-288f12166504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:53.540510 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628306-2b4c-4eec-9ae8-288f12166504 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:53.540791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628306-2b4c-4eec-9ae8-288f12166504 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:53.541064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628306-2b4c-4eec-9ae8-288f12166504 None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/0b380bc85da64a8bb687e2e5c4577e79/roles/609bba755f3f46f1b91a203b0a8ea404` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:53.544508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3628306-2b4c-4eec-9ae8-288f12166504 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=0b380bc85da64a8bb687e2e5c4577e79, role_id=609bba755f3f46f1b91a203b0a8ea404)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:53.545993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3628306-2b4c-4eec-9ae8-288f12166504 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:53.575639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbec49-be5c-4e4f-be1b-13d7e51f4e6a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:53.615067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 462/947] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:48:53 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/0b380bc85da64a8bb687e2e5c4577e79/roles/609bba755f3f46f1b91a203b0a8ea404 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:48:53.624734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0a45c-c7ae-435d-ac19-c93896839cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:53.626325 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a45c-c7ae-435d-ac19-c93896839cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:53.626688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a45c-c7ae-435d-ac19-c93896839cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:53.627028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a45c-c7ae-435d-ac19-c93896839cb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:53.635437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0a45c-c7ae-435d-ac19-c93896839cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:53.637284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0a45c-c7ae-435d-ac19-c93896839cb7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1fc0a45c-c7ae-435d-ac19-c93896839cb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:53.654857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:53.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:53.656755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:53.656755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:53.656755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:53.656755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:53.656755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:53.656755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:53.656755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:53.656755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:53.656755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 486/948] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:53.662797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628306-2b4c-4eec-9ae8-288f12166504 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:53.783115 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628306-2b4c-4eec-9ae8-288f12166504 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:53.826413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776c948-5433-4f87-a96d-dedfa65aabc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:53.827002 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776c948-5433-4f87-a96d-dedfa65aabc3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:53.827283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776c948-5433-4f87-a96d-dedfa65aabc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:53.827555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776c948-5433-4f87-a96d-dedfa65aabc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:53.828923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776c948-5433-4f87-a96d-dedfa65aabc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:53.829396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776c948-5433-4f87-a96d-dedfa65aabc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:53.830893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776c948-5433-4f87-a96d-dedfa65aabc3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:53.849748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 463/949] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:48:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:54.683753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855574c-8b92-469b-ab18-eac1001ef35b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:54.684304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855574c-8b92-469b-ab18-eac1001ef35b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:54.684304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855574c-8b92-469b-ab18-eac1001ef35b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:54.693786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 487/950] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:54.698248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d0ca4-e881-4ef6-a8e1-99f1578c779f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:54.698683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d0ca4-e881-4ef6-a8e1-99f1578c779f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:54.698867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d0ca4-e881-4ef6-a8e1-99f1578c779f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:54.716235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-197d0ca4-e881-4ef6-a8e1-99f1578c779f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:54.717398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-197d0ca4-e881-4ef6-a8e1-99f1578c779f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:54.743137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:54.798581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 464/951] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:54.819907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855574c-8b92-469b-ab18-eac1001ef35b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:54.931529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855574c-8b92-469b-ab18-eac1001ef35b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:54.941591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:54.967135 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8db0dc-9bd7-4c5f-833d-1827ddb857bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:54.967937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8db0dc-9bd7-4c5f-833d-1827ddb857bb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:54.968214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8db0dc-9bd7-4c5f-833d-1827ddb857bb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:54.968483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8db0dc-9bd7-4c5f-833d-1827ddb857bb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:54.969200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8db0dc-9bd7-4c5f-833d-1827ddb857bb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:54.970598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8db0dc-9bd7-4c5f-833d-1827ddb857bb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6c8db0dc-9bd7-4c5f-833d-1827ddb857bb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:54.982843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:54.983984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:54.985171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:54.986551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 488/952] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:48:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:54.991587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d0ca4-e881-4ef6-a8e1-99f1578c779f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:55.101888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d0ca4-e881-4ef6-a8e1-99f1578c779f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:55.138918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3482426-7815-403d-a9a7-01f18a9e42b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:55.139580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3482426-7815-403d-a9a7-01f18a9e42b2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:55.139858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3482426-7815-403d-a9a7-01f18a9e42b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:55.140134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3482426-7815-403d-a9a7-01f18a9e42b2 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:55.142095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3482426-7815-403d-a9a7-01f18a9e42b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:55.142410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3482426-7815-403d-a9a7-01f18a9e42b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:55.143769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3482426-7815-403d-a9a7-01f18a9e42b2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:55.153184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 465/953] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 13 08:48:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:55.155103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8db0dc-9bd7-4c5f-833d-1827ddb857bb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:55.227452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d51d73-742d-4a65-9f0f-26e2c47ddf96 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:55.227710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d51d73-742d-4a65-9f0f-26e2c47ddf96 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:55.227928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d51d73-742d-4a65-9f0f-26e2c47ddf96 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:55.232850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 466/954] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:55.235720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb26ad-aade-47c7-aeac-480f096100eb None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:55.236231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb26ad-aade-47c7-aeac-480f096100eb None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:55.236706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb26ad-aade-47c7-aeac-480f096100eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:55.255549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-2fcb26ad-aade-47c7-aeac-480f096100eb None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:55.257022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2fcb26ad-aade-47c7-aeac-480f096100eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:55.263873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8db0dc-9bd7-4c5f-833d-1827ddb857bb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:55.277432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:55.303951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc42cf-1b5d-40ad-ab2d-4838f87588d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:55.304988 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc42cf-1b5d-40ad-ab2d-4838f87588d0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:55.304988 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc42cf-1b5d-40ad-ab2d-4838f87588d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:55.305201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc42cf-1b5d-40ad-ab2d-4838f87588d0 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:55.305868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc42cf-1b5d-40ad-ab2d-4838f87588d0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:55.306219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc42cf-1b5d-40ad-ab2d-4838f87588d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:55.307703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc42cf-1b5d-40ad-ab2d-4838f87588d0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:55.317356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 489/955] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 13 08:48:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:55.321541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc42cf-1b5d-40ad-ab2d-4838f87588d0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:55.354221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 467/956] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:55.372338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb26ad-aade-47c7-aeac-480f096100eb None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:55.378893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:55.442624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc42cf-1b5d-40ad-ab2d-4838f87588d0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:55.484996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761798ce-446f-4c6a-b5fc-8560c9819d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:55.486482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761798ce-446f-4c6a-b5fc-8560c9819d3d None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:55.487311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761798ce-446f-4c6a-b5fc-8560c9819d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:55.488023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761798ce-446f-4c6a-b5fc-8560c9819d3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:55.489194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761798ce-446f-4c6a-b5fc-8560c9819d3d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:55.491459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761798ce-446f-4c6a-b5fc-8560c9819d3d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:55.492856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb26ad-aade-47c7-aeac-480f096100eb None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:55.530602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee56b2e-a350-4295-ba6a-79b569ba4fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:55.531898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee56b2e-a350-4295-ba6a-79b569ba4fbc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:55.531898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee56b2e-a350-4295-ba6a-79b569ba4fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:55.532255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee56b2e-a350-4295-ba6a-79b569ba4fbc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:55.533050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee56b2e-a350-4295-ba6a-79b569ba4fbc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:55.534506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee56b2e-a350-4295-ba6a-79b569ba4fbc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-4ee56b2e-a350-4295-ba6a-79b569ba4fbc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:55.547393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:55.550747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:55.558781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 468/957] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:55.559580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee56b2e-a350-4295-ba6a-79b569ba4fbc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:55.579464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:55.593844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 490/958] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 13 08:48:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:55.680702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee56b2e-a350-4295-ba6a-79b569ba4fbc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:55.725564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9565e6-af5d-44e1-a9d0-29d864eb0c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:55.726379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9565e6-af5d-44e1-a9d0-29d864eb0c33 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:55.726570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9565e6-af5d-44e1-a9d0-29d864eb0c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:55.726799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9565e6-af5d-44e1-a9d0-29d864eb0c33 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:55.727503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9565e6-af5d-44e1-a9d0-29d864eb0c33 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:55.727805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9565e6-af5d-44e1-a9d0-29d864eb0c33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:55.729168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9565e6-af5d-44e1-a9d0-29d864eb0c33 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:55.735351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 469/959] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:48:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:55.743933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761798ce-446f-4c6a-b5fc-8560c9819d3d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:55.785306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a62c9-5383-4d1e-9d15-8a0887ff74b0 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:55.785640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a62c9-5383-4d1e-9d15-8a0887ff74b0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:55.785983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a62c9-5383-4d1e-9d15-8a0887ff74b0 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:55.787369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 470/960] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:55.793328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c483-0740-47f5-80e5-3916edd54517 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:55.793605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c483-0740-47f5-80e5-3916edd54517 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:55.793884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c483-0740-47f5-80e5-3916edd54517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:55.811242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a914c483-0740-47f5-80e5-3916edd54517 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:55.812423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a914c483-0740-47f5-80e5-3916edd54517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:55.863352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:55.873867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761798ce-446f-4c6a-b5fc-8560c9819d3d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:55.890079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 471/961] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:55.920481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914c483-0740-47f5-80e5-3916edd54517 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:55.924047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc9aef-336e-4f97-94a0-1bfb7e318dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:55.925388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc9aef-336e-4f97-94a0-1bfb7e318dad None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:55.925388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc9aef-336e-4f97-94a0-1bfb7e318dad None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:55.925666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc9aef-336e-4f97-94a0-1bfb7e318dad None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:55.926529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc9aef-336e-4f97-94a0-1bfb7e318dad None admin] RBAC: Adding query filter params (type=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:55.926974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc9aef-336e-4f97-94a0-1bfb7e318dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:55.928446 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc9aef-336e-4f97-94a0-1bfb7e318dad None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:55.933658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:55.942260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 491/962] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:48:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:56.018907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf6954-5a38-466a-bf63-fa4886258729 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:56.019111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf6954-5a38-466a-bf63-fa4886258729 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:56.019377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf6954-5a38-466a-bf63-fa4886258729 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:56.021037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 492/963] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:48:56.026503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764cdfe-8bb3-4b5b-b462-7cffdc2d3bc8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:56.026782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764cdfe-8bb3-4b5b-b462-7cffdc2d3bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:56.027066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764cdfe-8bb3-4b5b-b462-7cffdc2d3bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:56.037930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914c483-0740-47f5-80e5-3916edd54517 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:56.045899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2764cdfe-8bb3-4b5b-b462-7cffdc2d3bc8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:56.047596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2764cdfe-8bb3-4b5b-b462-7cffdc2d3bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:56.081352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa8861-c00d-40e1-8824-7307571ebd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:56.082229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa8861-c00d-40e1-8824-7307571ebd03 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:56.082516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa8861-c00d-40e1-8824-7307571ebd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:56.082791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa8861-c00d-40e1-8824-7307571ebd03 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:56.096257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa8861-c00d-40e1-8824-7307571ebd03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-5ffa8861-c00d-40e1-8824-7307571ebd03 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:48:56.105209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:56.106264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:56.107622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 472/964] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 13 08:48:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:56.115403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa8861-c00d-40e1-8824-7307571ebd03 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:56.141780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 493/965] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:56.171276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764cdfe-8bb3-4b5b-b462-7cffdc2d3bc8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:56.198980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:56.243852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa8861-c00d-40e1-8824-7307571ebd03 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:56.267211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:56.286656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764cdfe-8bb3-4b5b-b462-7cffdc2d3bc8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:56.292023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ec594-37bb-4491-bb33-6e155199fb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:56.292987 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ec594-37bb-4491-bb33-6e155199fb21 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:56.293893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ec594-37bb-4491-bb33-6e155199fb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:56.294668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ec594-37bb-4491-bb33-6e155199fb21 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:56.295652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ec594-37bb-4491-bb33-6e155199fb21 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:56.296484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ec594-37bb-4491-bb33-6e155199fb21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:56.298758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ec594-37bb-4491-bb33-6e155199fb21 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:56.310812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 473/966] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 13 08:48:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:56.316549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ec594-37bb-4491-bb33-6e155199fb21 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:56.330415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec2279-4822-481c-b863-1ae56494aefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:56.331098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec2279-4822-481c-b863-1ae56494aefe None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:56.331377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec2279-4822-481c-b863-1ae56494aefe None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:56.331706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec2279-4822-481c-b863-1ae56494aefe None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:56.332409 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec2279-4822-481c-b863-1ae56494aefe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:56.334571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec2279-4822-481c-b863-1ae56494aefe None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:56.379290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.sql.core [None req-38ec2279-4822-481c-b863-1ae56494aefe None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 08:48:56.379290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 13 08:48:56.379290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [parameters: {'user_id': '1bcd08e045f14a1881fbbd42255dc310', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 13 08:48:56.379290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80287) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-38ec2279-4822-481c-b863-1ae56494aefe None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application conn.query(q) Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result.read() Jul 13 08:48:56.387472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 08:48:56.388369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application next(self.gen) Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application yield resource Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application next(self.gen) Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise self.value Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application session.commit() Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 08:48:56.389385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.session.flush() Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application compat.raise_( Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 08:48:56.390469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 08:48:56.392059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application conn.query(q) Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result.read() Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 08:48:56.392921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [parameters: {'user_id': '1bcd08e045f14a1881fbbd42255dc310', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:56.393769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 13 08:48:56.396673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 13 08:48:56.398912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 13 08:48:56.398912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 13 08:48:56.398912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 08:48:56.398912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 08:48:56.398912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 13 08:48:56.398912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:56.398912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 494/967] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 13 08:48:56 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:56.398912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec2279-4822-481c-b863-1ae56494aefe None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:56.449005 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ec594-37bb-4491-bb33-6e155199fb21 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:56.479758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:56.486504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78da24-6cd6-46ab-ab10-ec6a5941b30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:56.487585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78da24-6cd6-46ab-ab10-ec6a5941b30b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:56.488093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78da24-6cd6-46ab-ab10-ec6a5941b30b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:56.488528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78da24-6cd6-46ab-ab10-ec6a5941b30b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:56.501104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78da24-6cd6-46ab-ab10-ec6a5941b30b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:56.502882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78da24-6cd6-46ab-ab10-ec6a5941b30b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:56.514088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec2279-4822-481c-b863-1ae56494aefe None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-1c78da24-6cd6-46ab-ab10-ec6a5941b30b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:56.522149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:56.523068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:56.524058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:56.524058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:56.524058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:56.524058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:56.526272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 474/968] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:56.533160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78da24-6cd6-46ab-ab10-ec6a5941b30b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:56.564372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9632f-2be8-41f4-967f-65b1c8944543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:56.566404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9632f-2be8-41f4-967f-65b1c8944543 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:56.567159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9632f-2be8-41f4-967f-65b1c8944543 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:56.568029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9632f-2be8-41f4-967f-65b1c8944543 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:56.579272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9632f-2be8-41f4-967f-65b1c8944543 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:56.581672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9632f-2be8-41f4-967f-65b1c8944543 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4ec9632f-2be8-41f4-967f-65b1c8944543 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:56.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:56.602662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:56.603631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:56.603631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:56.603631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:56.603631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:56.603631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:56.603631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:56.603631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:56.603631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:56.603631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 08:48:56.603631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:56.607267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 495/969] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:48:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:56.612727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9632f-2be8-41f4-967f-65b1c8944543 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:56.625937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:56.654216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78da24-6cd6-46ab-ab10-ec6a5941b30b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:56.691624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c5c46-c111-41ab-a2d9-889916fddadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:56.692311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c5c46-c111-41ab-a2d9-889916fddadc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:56.692590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c5c46-c111-41ab-a2d9-889916fddadc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:56.692906 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c5c46-c111-41ab-a2d9-889916fddadc None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:56.693621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c5c46-c111-41ab-a2d9-889916fddadc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:56.694089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c5c46-c111-41ab-a2d9-889916fddadc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:56.695664 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c5c46-c111-41ab-a2d9-889916fddadc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:56.715035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 475/970] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:56.716343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c5c46-c111-41ab-a2d9-889916fddadc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:56.730211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9632f-2be8-41f4-967f-65b1c8944543 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:56.763859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:56.769253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96497d-9517-4928-af76-595a65cfaf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:56.770485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96497d-9517-4928-af76-595a65cfaf16 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:56.770970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96497d-9517-4928-af76-595a65cfaf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:56.781286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96497d-9517-4928-af76-595a65cfaf16 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:56.781286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96497d-9517-4928-af76-595a65cfaf16 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:56.781286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96497d-9517-4928-af76-595a65cfaf16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:56.781286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96497d-9517-4928-af76-595a65cfaf16 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:56.815615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 496/971] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 13 08:48:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:56.833908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c5c46-c111-41ab-a2d9-889916fddadc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:56.876641 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8eb16-e589-4c87-8323-d782df7d1329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:56.877944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8eb16-e589-4c87-8323-d782df7d1329 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:56.878446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8eb16-e589-4c87-8323-d782df7d1329 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:56.878964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8eb16-e589-4c87-8323-d782df7d1329 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:56.887922 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8eb16-e589-4c87-8323-d782df7d1329 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:56.889846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8eb16-e589-4c87-8323-d782df7d1329 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d4c8eb16-e589-4c87-8323-d782df7d1329 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:48:56.908798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:56.909871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:56.910851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:56.910851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:56.910851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:48:56.910851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:48:56.910851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:48:56.910851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:48:56.910851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:48:56.910851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:48:56.910851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 08:48:56.910851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:56.913676 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 476/972] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 13 08:48:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:56.919615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96497d-9517-4928-af76-595a65cfaf16 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:56.998357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:57.036157 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96497d-9517-4928-af76-595a65cfaf16 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:57.074789 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e8431-d18a-4b19-9be9-d8a711c80c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:57.075390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e8431-d18a-4b19-9be9-d8a711c80c17 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:57.075669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e8431-d18a-4b19-9be9-d8a711c80c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:57.075941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e8431-d18a-4b19-9be9-d8a711c80c17 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:57.076811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e8431-d18a-4b19-9be9-d8a711c80c17 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:57.077116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e8431-d18a-4b19-9be9-d8a711c80c17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:57.079311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e8431-d18a-4b19-9be9-d8a711c80c17 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:57.110459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 497/973] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 13 08:48:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:57.113624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8eb16-e589-4c87-8323-d782df7d1329 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:57.216524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8eb16-e589-4c87-8323-d782df7d1329 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:57.218072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:57.252067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2452a-15fe-4a05-ac66-505c18eedf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:57.252787 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2452a-15fe-4a05-ac66-505c18eedf94 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:57.253130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2452a-15fe-4a05-ac66-505c18eedf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:57.253466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2452a-15fe-4a05-ac66-505c18eedf94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:57.320743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2452a-15fe-4a05-ac66-505c18eedf94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:57.322298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2452a-15fe-4a05-ac66-505c18eedf94 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:57.432838 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 477/974] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:48:57 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:48:57.434060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e8431-d18a-4b19-9be9-d8a711c80c17 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:57.534446 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e8431-d18a-4b19-9be9-d8a711c80c17 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:57.558616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:57.570329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ad7be-6037-4ca5-b234-4415fa4619dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:57.571480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ad7be-6037-4ca5-b234-4415fa4619dc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:57.571588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ad7be-6037-4ca5-b234-4415fa4619dc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:57.571826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ad7be-6037-4ca5-b234-4415fa4619dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:57.581735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ad7be-6037-4ca5-b234-4415fa4619dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:57.583250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ad7be-6037-4ca5-b234-4415fa4619dc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a29ad7be-6037-4ca5-b234-4415fa4619dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:48:57.600724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:57.601630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:57.607035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:48:57.607035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:48:57.607035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:48:57.607035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:57.607035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 498/975] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:57.608996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2452a-15fe-4a05-ac66-505c18eedf94 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:57.611795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:57.711396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2452a-15fe-4a05-ac66-505c18eedf94 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:57.745529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828b5f2-8151-4893-951a-384629b90d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:57.746229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b5f2-8151-4893-951a-384629b90d81 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:57.746514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b5f2-8151-4893-951a-384629b90d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:57.746789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b5f2-8151-4893-951a-384629b90d81 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:57.747505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828b5f2-8151-4893-951a-384629b90d81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:57.747809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828b5f2-8151-4893-951a-384629b90d81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:57.749166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828b5f2-8151-4893-951a-384629b90d81 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:57.766316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 478/976] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:57.768060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ad7be-6037-4ca5-b234-4415fa4619dc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:57.782797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40394f2a-f417-446f-980a-46a3b96b7612 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:57.783203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40394f2a-f417-446f-980a-46a3b96b7612 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:57.783487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40394f2a-f417-446f-980a-46a3b96b7612 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:57.784954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 479/977] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:57.789700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858ad5d-c5a6-4182-8e16-55800b226af2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:57.789985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858ad5d-c5a6-4182-8e16-55800b226af2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:57.790296 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858ad5d-c5a6-4182-8e16-55800b226af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:57.807646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9858ad5d-c5a6-4182-8e16-55800b226af2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:57.808822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9858ad5d-c5a6-4182-8e16-55800b226af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:57.866574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:57.877800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 480/978] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:48:57.880709 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ad7be-6037-4ca5-b234-4415fa4619dc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:57.900856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858ad5d-c5a6-4182-8e16-55800b226af2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:57.917803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92322-7f58-4d24-a0e4-bcaf00d409e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:57.918646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92322-7f58-4d24-a0e4-bcaf00d409e8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:57.918930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92322-7f58-4d24-a0e4-bcaf00d409e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:57.919459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92322-7f58-4d24-a0e4-bcaf00d409e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:57.927130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac92322-7f58-4d24-a0e4-bcaf00d409e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:57.928628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac92322-7f58-4d24-a0e4-bcaf00d409e8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-dac92322-7f58-4d24-a0e4-bcaf00d409e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:48:57.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:57.945037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:57.946701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:48:57.946701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:48:57.946701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:48:57.946701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:48:57.946701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:48:57.946701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:48:57.946701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:48:57.946701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:48:57.946701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 499/979] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:48:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:48:57.951173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92322-7f58-4d24-a0e4-bcaf00d409e8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:58.000179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:58.009075 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858ad5d-c5a6-4182-8e16-55800b226af2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:58.045775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fa356-617a-4b8c-be28-89ba5bc512d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:58.046609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fa356-617a-4b8c-be28-89ba5bc512d4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:58.048000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fa356-617a-4b8c-be28-89ba5bc512d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:58.048280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fa356-617a-4b8c-be28-89ba5bc512d4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:58.049006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fa356-617a-4b8c-be28-89ba5bc512d4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:58.050384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fa356-617a-4b8c-be28-89ba5bc512d4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-528fa356-617a-4b8c-be28-89ba5bc512d4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:48:58.062943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:48:58.064166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:48:58.068401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 481/980] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:48:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:48:58.069299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d004ab-b990-45dc-b6f7-950b4dd9d1dd None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:58.069299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d004ab-b990-45dc-b6f7-950b4dd9d1dd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:58.069299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d004ab-b990-45dc-b6f7-950b4dd9d1dd None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:58.069299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 482/981] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:48:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:48:58.070386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d004ab-b990-45dc-b6f7-950b4dd9d1dd None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:58.072009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92322-7f58-4d24-a0e4-bcaf00d409e8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:58.106329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc579fc1-942a-402f-a543-814bddfc72bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:58.106962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579fc1-942a-402f-a543-814bddfc72bb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:58.107243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579fc1-942a-402f-a543-814bddfc72bb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:58.107510 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579fc1-942a-402f-a543-814bddfc72bb None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:58.108291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc579fc1-942a-402f-a543-814bddfc72bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:58.108595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc579fc1-942a-402f-a543-814bddfc72bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:58.111117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc579fc1-942a-402f-a543-814bddfc72bb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:58.122577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 500/982] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:48:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:58.123845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fff10-ccad-41af-9a19-350b7afdb000 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:58.124134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fff10-ccad-41af-9a19-350b7afdb000 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:58.124411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fff10-ccad-41af-9a19-350b7afdb000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:58.128988 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:58.140749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-352fff10-ccad-41af-9a19-350b7afdb000 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:48:58.142411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-352fff10-ccad-41af-9a19-350b7afdb000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:48:58.175236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d004ab-b990-45dc-b6f7-950b4dd9d1dd None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:58.207891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5f0ac-bf16-484a-bac9-3d38ca134c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:58.208551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5f0ac-bf16-484a-bac9-3d38ca134c1f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:58.208827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 501/983] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:48:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:48:58.208926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5f0ac-bf16-484a-bac9-3d38ca134c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:58.209101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5f0ac-bf16-484a-bac9-3d38ca134c1f None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:58.209646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fff10-ccad-41af-9a19-350b7afdb000 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:58.209802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5f0ac-bf16-484a-bac9-3d38ca134c1f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:58.210139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5f0ac-bf16-484a-bac9-3d38ca134c1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:58.211532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5f0ac-bf16-484a-bac9-3d38ca134c1f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:58.216372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 483/984] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 13 08:48:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:58.222358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5f0ac-bf16-484a-bac9-3d38ca134c1f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:58.242512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:58.315397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fff10-ccad-41af-9a19-350b7afdb000 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:58.324233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5f0ac-bf16-484a-bac9-3d38ca134c1f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:58.348204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:58.353494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5c6a9-f135-4b47-9b94-d3c39bbe9560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:58.354186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5c6a9-f135-4b47-9b94-d3c39bbe9560 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:58.354467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5c6a9-f135-4b47-9b94-d3c39bbe9560 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:58.354737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5c6a9-f135-4b47-9b94-d3c39bbe9560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:58.355569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5c6a9-f135-4b47-9b94-d3c39bbe9560 None admin] RBAC: Adding query filter params (role.id=609bba755f3f46f1b91a203b0a8ea404, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=0b380bc85da64a8bb687e2e5c4577e79) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:58.355879 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5c6a9-f135-4b47-9b94-d3c39bbe9560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:58.357309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5c6a9-f135-4b47-9b94-d3c39bbe9560 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:58.362052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348aa6a-89bb-478e-a9ed-97ea919f6a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:58.362700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348aa6a-89bb-478e-a9ed-97ea919f6a99 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:58.362977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348aa6a-89bb-478e-a9ed-97ea919f6a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:58.363277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348aa6a-89bb-478e-a9ed-97ea919f6a99 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:58.363689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 502/985] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:48:58 2022] GET /identity/v3/role_assignments?user.id=0b380bc85da64a8bb687e2e5c4577e79&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=609bba755f3f46f1b91a203b0a8ea404 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:48:58.364055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348aa6a-89bb-478e-a9ed-97ea919f6a99 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:58.364269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5c6a9-f135-4b47-9b94-d3c39bbe9560 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:58.364410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348aa6a-89bb-478e-a9ed-97ea919f6a99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:58.366134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348aa6a-89bb-478e-a9ed-97ea919f6a99 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:58.370963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 484/986] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 13 08:48:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:58.377936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348aa6a-89bb-478e-a9ed-97ea919f6a99 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:58.471922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:58.481912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5c6a9-f135-4b47-9b94-d3c39bbe9560 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:58.494815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348aa6a-89bb-478e-a9ed-97ea919f6a99 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:58.523846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5a3a7-203c-4584-8b4a-de3ce3b1de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:58.524838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5a3a7-203c-4584-8b4a-de3ce3b1de8d None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:58.525283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5a3a7-203c-4584-8b4a-de3ce3b1de8d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:58.525566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5a3a7-203c-4584-8b4a-de3ce3b1de8d None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:58.526323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5a3a7-203c-4584-8b4a-de3ce3b1de8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:58.527696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5a3a7-203c-4584-8b4a-de3ce3b1de8d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:58.535657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4956ca-0633-43b9-b569-ead8e08ca272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:58.536333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4956ca-0633-43b9-b569-ead8e08ca272 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:58.536617 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4956ca-0633-43b9-b569-ead8e08ca272 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:58.536892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4956ca-0633-43b9-b569-ead8e08ca272 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:58.537696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4956ca-0633-43b9-b569-ead8e08ca272 None admin] RBAC: Adding query filter params (interface=admin, service_id=9355914168f140eba5a39cacfed3333b, region_id=RegionOne) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:48:58.538010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4956ca-0633-43b9-b569-ead8e08ca272 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:58.539441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4956ca-0633-43b9-b569-ead8e08ca272 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:58.547975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 485/987] 10.222.0.40 () {56 vars in 1249 bytes} [Wed Jul 13 08:48:58 2022] GET /identity/v3/endpoints?service_id=9355914168f140eba5a39cacfed3333b&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:48:58.553543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4956ca-0633-43b9-b569-ead8e08ca272 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:48:58.599894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 503/988] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:48:58 2022] POST /identity/v3/services => generated 242 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:48:58.604079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:48:58.664814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4956ca-0633-43b9-b569-ead8e08ca272 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:48:58.709852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096401e-2ec8-4b44-9386-ce08696ed607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:48:58.710561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096401e-2ec8-4b44-9386-ce08696ed607 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:48:58.710844 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096401e-2ec8-4b44-9386-ce08696ed607 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:48:58.711115 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096401e-2ec8-4b44-9386-ce08696ed607 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:48:58.711816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096401e-2ec8-4b44-9386-ce08696ed607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:48:58.713195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096401e-2ec8-4b44-9386-ce08696ed607 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:48:58.726434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 486/989] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 13 08:48:58 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 08:49:00.449753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ed168-2159-4240-a386-295b1d03e022 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:00.450306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ed168-2159-4240-a386-295b1d03e022 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:00.450306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ed168-2159-4240-a386-295b1d03e022 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:00.451815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 504/990] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:00.457580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a842f2-1625-4ee1-9d2c-c93b5e2ed2e2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:00.457990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a842f2-1625-4ee1-9d2c-c93b5e2ed2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:00.458189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a842f2-1625-4ee1-9d2c-c93b5e2ed2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:00.475307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c5a842f2-1625-4ee1-9d2c-c93b5e2ed2e2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:00.476489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c5a842f2-1625-4ee1-9d2c-c93b5e2ed2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:00.548440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 487/991] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:00.570776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ed168-2159-4240-a386-295b1d03e022 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:00.669610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ca83f-1d80-407e-b669-30711114f3e4 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:00.669904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ca83f-1d80-407e-b669-30711114f3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:00.670267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ca83f-1d80-407e-b669-30711114f3e4 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:00.674085 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 488/992] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:49:00.676417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ed168-2159-4240-a386-295b1d03e022 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:00.677565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d521d84-9fcc-4cfd-b6e8-c61f4aba4f9e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:00.677886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d521d84-9fcc-4cfd-b6e8-c61f4aba4f9e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:00.678214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d521d84-9fcc-4cfd-b6e8-c61f4aba4f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:00.695630 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4d521d84-9fcc-4cfd-b6e8-c61f4aba4f9e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:00.696822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4d521d84-9fcc-4cfd-b6e8-c61f4aba4f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:00.712292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab20b1-d738-476a-a99f-90d21141c61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:00.713073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab20b1-d738-476a-a99f-90d21141c61a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:00.713386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab20b1-d738-476a-a99f-90d21141c61a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:00.713644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab20b1-d738-476a-a99f-90d21141c61a None admin] PATH_INFO: `/v3/services/image` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:00.714356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab20b1-d738-476a-a99f-90d21141c61a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:00.715857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab20b1-d738-476a-a99f-90d21141c61a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-92ab20b1-d738-476a-a99f-90d21141c61a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:00.727885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:00.728872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:00.729706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 505/993] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Jul 13 08:49:00 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:00.732059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:00.735621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab20b1-d738-476a-a99f-90d21141c61a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:00.752618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:00.770681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 489/994] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:49:00.793182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbacc67-f6b9-4afe-b35e-6fa31a829b96 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:00.793469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbacc67-f6b9-4afe-b35e-6fa31a829b96 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:00.793749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbacc67-f6b9-4afe-b35e-6fa31a829b96 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:00.795385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 490/995] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:49:00.796069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbacc67-f6b9-4afe-b35e-6fa31a829b96 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:00.838320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab20b1-d738-476a-a99f-90d21141c61a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:00.873434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edb171-16fa-4f09-969f-3a1ee15e81e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:00.874126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edb171-16fa-4f09-969f-3a1ee15e81e1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:00.874412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edb171-16fa-4f09-969f-3a1ee15e81e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:00.874712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edb171-16fa-4f09-969f-3a1ee15e81e1 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:00.876314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edb171-16fa-4f09-969f-3a1ee15e81e1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:00.876455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edb171-16fa-4f09-969f-3a1ee15e81e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:00.877747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edb171-16fa-4f09-969f-3a1ee15e81e1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:00.882649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 506/996] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:00.883805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305d248-a59d-4498-8e9c-7302fb007e72 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:00.884095 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305d248-a59d-4498-8e9c-7302fb007e72 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:00.884373 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305d248-a59d-4498-8e9c-7302fb007e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:00.897666 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbacc67-f6b9-4afe-b35e-6fa31a829b96 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:00.900945 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2305d248-a59d-4498-8e9c-7302fb007e72 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:00.902145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2305d248-a59d-4498-8e9c-7302fb007e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:00.933009 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281a96f-19d2-42de-acaa-ec3d1a02617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:00.933727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281a96f-19d2-42de-acaa-ec3d1a02617e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:00.934094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281a96f-19d2-42de-acaa-ec3d1a02617e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:00.934378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281a96f-19d2-42de-acaa-ec3d1a02617e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:00.935030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281a96f-19d2-42de-acaa-ec3d1a02617e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:00.936384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281a96f-19d2-42de-acaa-ec3d1a02617e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3281a96f-19d2-42de-acaa-ec3d1a02617e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:00.948298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:00.949333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:00.950953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 491/997] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:49:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:00.951765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281a96f-19d2-42de-acaa-ec3d1a02617e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:00.970433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 507/998] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:00.971269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305d248-a59d-4498-8e9c-7302fb007e72 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:00.981886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:00.986231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:01.054128 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281a96f-19d2-42de-acaa-ec3d1a02617e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:01.072292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305d248-a59d-4498-8e9c-7302fb007e72 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:01.087697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be010960-0748-4bee-9d59-4c2ea22e5941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:01.088346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be010960-0748-4bee-9d59-4c2ea22e5941 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:01.088622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be010960-0748-4bee-9d59-4c2ea22e5941 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:01.088899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be010960-0748-4bee-9d59-4c2ea22e5941 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:01.089586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be010960-0748-4bee-9d59-4c2ea22e5941 None admin] RBAC: Adding query filter params (type=image) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:01.089884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be010960-0748-4bee-9d59-4c2ea22e5941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:01.091260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be010960-0748-4bee-9d59-4c2ea22e5941 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:01.096446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 492/999] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:01.097137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be010960-0748-4bee-9d59-4c2ea22e5941 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:01.109977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d66af-1b01-4cb6-a20c-0c3a9455833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:01.110690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d66af-1b01-4cb6-a20c-0c3a9455833d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:01.110981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d66af-1b01-4cb6-a20c-0c3a9455833d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:01.111260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d66af-1b01-4cb6-a20c-0c3a9455833d None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:01.111995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d66af-1b01-4cb6-a20c-0c3a9455833d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:01.112306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d66af-1b01-4cb6-a20c-0c3a9455833d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:01.113746 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d66af-1b01-4cb6-a20c-0c3a9455833d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:01.119712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 508/1000] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:49:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:01.125818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d66af-1b01-4cb6-a20c-0c3a9455833d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:01.204135 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:01.219667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:01.226672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be010960-0748-4bee-9d59-4c2ea22e5941 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:01.236489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d66af-1b01-4cb6-a20c-0c3a9455833d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:01.268572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8df85d-d6a7-4014-9383-b97547a612b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:01.269415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8df85d-d6a7-4014-9383-b97547a612b6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:01.269705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8df85d-d6a7-4014-9383-b97547a612b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:01.271245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8df85d-d6a7-4014-9383-b97547a612b6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:01.271245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8df85d-d6a7-4014-9383-b97547a612b6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:01.272125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8df85d-d6a7-4014-9383-b97547a612b6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:01.276592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e02e9e-f59a-495d-956a-00760a93aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:01.277708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e02e9e-f59a-495d-956a-00760a93aa0a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:01.277708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e02e9e-f59a-495d-956a-00760a93aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:01.277905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e02e9e-f59a-495d-956a-00760a93aa0a None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:01.278476 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e02e9e-f59a-495d-956a-00760a93aa0a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:01.278780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e02e9e-f59a-495d-956a-00760a93aa0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:01.280128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e02e9e-f59a-495d-956a-00760a93aa0a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ea8df85d-d6a7-4014-9383-b97547a612b6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:01.285547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:01.286752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 08:49:01.287611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:01.288425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 509/1001] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:49:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:01.290318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 493/1002] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 13 08:49:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:01.291174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e02e9e-f59a-495d-956a-00760a93aa0a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:01.292754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8df85d-d6a7-4014-9383-b97547a612b6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:01.401919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e02e9e-f59a-495d-956a-00760a93aa0a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:01.404567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8df85d-d6a7-4014-9383-b97547a612b6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:01.439460 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e2ab3-96a1-47f8-8b2f-eabc27ebe075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:01.440115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e2ab3-96a1-47f8-8b2f-eabc27ebe075 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:01.440394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e2ab3-96a1-47f8-8b2f-eabc27ebe075 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:01.440665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e2ab3-96a1-47f8-8b2f-eabc27ebe075 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:01.441549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e2ab3-96a1-47f8-8b2f-eabc27ebe075 None admin] RBAC: Adding query filter params (interface=public, service_id=d388d102280c49bdbf82d5bccf952e5c, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:01.441958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e2ab3-96a1-47f8-8b2f-eabc27ebe075 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:01.442982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083393d8-d6c8-4c00-b003-9d4ad9e28c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:01.443434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e2ab3-96a1-47f8-8b2f-eabc27ebe075 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:01.443646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083393d8-d6c8-4c00-b003-9d4ad9e28c90 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:01.443933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083393d8-d6c8-4c00-b003-9d4ad9e28c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:01.444214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083393d8-d6c8-4c00-b003-9d4ad9e28c90 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:01.445035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083393d8-d6c8-4c00-b003-9d4ad9e28c90 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:01.445348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083393d8-d6c8-4c00-b003-9d4ad9e28c90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:01.445980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:01.446743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083393d8-d6c8-4c00-b003-9d4ad9e28c90 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:01.449614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 510/1003] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 13 08:49:01 2022] GET /identity/v3/endpoints?service_id=d388d102280c49bdbf82d5bccf952e5c&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:01.459517 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e2ab3-96a1-47f8-8b2f-eabc27ebe075 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:01.495363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 494/1004] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 13 08:49:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:01.501517 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083393d8-d6c8-4c00-b003-9d4ad9e28c90 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:01.528238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:01.563413 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e2ab3-96a1-47f8-8b2f-eabc27ebe075 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:01.601917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818ead9-ad15-473f-9d29-b8403c89c0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:01.602616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818ead9-ad15-473f-9d29-b8403c89c0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:01.602894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818ead9-ad15-473f-9d29-b8403c89c0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:01.603187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818ead9-ad15-473f-9d29-b8403c89c0d1 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:01.604048 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818ead9-ad15-473f-9d29-b8403c89c0d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:01.605268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818ead9-ad15-473f-9d29-b8403c89c0d1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:01.611858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083393d8-d6c8-4c00-b003-9d4ad9e28c90 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:01.615463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 511/1005] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 13 08:49:01 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 08:49:01.649082 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dd425-b757-438f-9d90-83dc5ebd2cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:01.649765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dd425-b757-438f-9d90-83dc5ebd2cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:01.650066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dd425-b757-438f-9d90-83dc5ebd2cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:01.650439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dd425-b757-438f-9d90-83dc5ebd2cd8 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:01.651167 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dd425-b757-438f-9d90-83dc5ebd2cd8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:01.651475 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dd425-b757-438f-9d90-83dc5ebd2cd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:01.652881 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dd425-b757-438f-9d90-83dc5ebd2cd8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:01.664070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 495/1006] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 13 08:49:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:01.665099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818ead9-ad15-473f-9d29-b8403c89c0d1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:01.735044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:01.777765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818ead9-ad15-473f-9d29-b8403c89c0d1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:01.815397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe616167-8930-42a0-b4b3-98b9eda082a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:01.816057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe616167-8930-42a0-b4b3-98b9eda082a0 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:01.816334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe616167-8930-42a0-b4b3-98b9eda082a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:01.816605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe616167-8930-42a0-b4b3-98b9eda082a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:01.817363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe616167-8930-42a0-b4b3-98b9eda082a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:01.818776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe616167-8930-42a0-b4b3-98b9eda082a0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:01.891179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 512/1007] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 13 08:49:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:49:03.128190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6fa60-7651-415b-9d7e-5bc91e5faebd None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:03.128720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6fa60-7651-415b-9d7e-5bc91e5faebd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:03.128720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6fa60-7651-415b-9d7e-5bc91e5faebd None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:03.130346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 496/1008] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:03.135544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdb748-a672-4f8e-8d12-dc25499285da None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:03.135955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdb748-a672-4f8e-8d12-dc25499285da None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:03.136095 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdb748-a672-4f8e-8d12-dc25499285da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:03.154519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4ffdb748-a672-4f8e-8d12-dc25499285da None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:03.155684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4ffdb748-a672-4f8e-8d12-dc25499285da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:03.236089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 513/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:03 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:03.258743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6fa60-7651-415b-9d7e-5bc91e5faebd None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:03.368870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:03.383203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6fa60-7651-415b-9d7e-5bc91e5faebd None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:03.424651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127691f-9a4a-492c-9bf4-095a159cda47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:03.425327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127691f-9a4a-492c-9bf4-095a159cda47 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:03.425611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127691f-9a4a-492c-9bf4-095a159cda47 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:03.425888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127691f-9a4a-492c-9bf4-095a159cda47 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:03.426710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127691f-9a4a-492c-9bf4-095a159cda47 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:03.428131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127691f-9a4a-492c-9bf4-095a159cda47 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:03.534066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 497/1010] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:49:03 2022] POST /identity/v3/services => generated 242 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:49:03.727198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1d08a-7193-4c23-9e92-e8103a6531e9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:03.727385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1d08a-7193-4c23-9e92-e8103a6531e9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:03.727701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1d08a-7193-4c23-9e92-e8103a6531e9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:03.729250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 514/1011] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:03.735680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d55de-9eec-415a-9e99-2920a4ee03a1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:03.736071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d55de-9eec-415a-9e99-2920a4ee03a1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:03.736359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d55de-9eec-415a-9e99-2920a4ee03a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:03.756799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a31d55de-9eec-415a-9e99-2920a4ee03a1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:03.758002 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a31d55de-9eec-415a-9e99-2920a4ee03a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:03.841814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 498/1012] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:03 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:03.871439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1d08a-7193-4c23-9e92-e8103a6531e9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:03.940871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:03.995990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ddc73-12bc-4d30-af46-716ebe709b81 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:03.996273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ddc73-12bc-4d30-af46-716ebe709b81 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:03.996560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ddc73-12bc-4d30-af46-716ebe709b81 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:03.998077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 499/1013] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:49:04.004091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76306408-cebd-4077-ba45-8e93c29c8fc7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:04.004440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76306408-cebd-4077-ba45-8e93c29c8fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:04.004653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76306408-cebd-4077-ba45-8e93c29c8fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:04.005618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1d08a-7193-4c23-9e92-e8103a6531e9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:04.021769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-76306408-cebd-4077-ba45-8e93c29c8fc7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:04.022997 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-76306408-cebd-4077-ba45-8e93c29c8fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:04.048614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fd189-2a60-4ae7-bca9-376e64e724e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:04.051801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd189-2a60-4ae7-bca9-376e64e724e2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:04.051801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd189-2a60-4ae7-bca9-376e64e724e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:04.051801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd189-2a60-4ae7-bca9-376e64e724e2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:04.051801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fd189-2a60-4ae7-bca9-376e64e724e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:04.052076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fd189-2a60-4ae7-bca9-376e64e724e2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-217fd189-2a60-4ae7-bca9-376e64e724e2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:04.065000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:04.066116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:04.066990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 515/1014] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:49:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:04.073084 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fd189-2a60-4ae7-bca9-376e64e724e2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:04.101142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 500/1015] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:04 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:49:04.125150 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76306408-cebd-4077-ba45-8e93c29c8fc7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:04.130281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:04.183114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fd189-2a60-4ae7-bca9-376e64e724e2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:04.197172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:04.217419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f396c51-88ee-46dc-94a8-812a2ea458a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:04.218121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f396c51-88ee-46dc-94a8-812a2ea458a1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:04.218960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f396c51-88ee-46dc-94a8-812a2ea458a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:04.219378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f396c51-88ee-46dc-94a8-812a2ea458a1 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:04.220296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f396c51-88ee-46dc-94a8-812a2ea458a1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:04.220630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f396c51-88ee-46dc-94a8-812a2ea458a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:04.222046 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f396c51-88ee-46dc-94a8-812a2ea458a1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:04.227150 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 516/1016] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:49:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:04.232765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f396c51-88ee-46dc-94a8-812a2ea458a1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:04.242935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76306408-cebd-4077-ba45-8e93c29c8fc7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:04.285690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40028c46-fbb2-4d61-9aec-83519df0923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:04.286433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40028c46-fbb2-4d61-9aec-83519df0923c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:04.286724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40028c46-fbb2-4d61-9aec-83519df0923c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:04.286978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40028c46-fbb2-4d61-9aec-83519df0923c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:04.288138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40028c46-fbb2-4d61-9aec-83519df0923c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:04.289549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40028c46-fbb2-4d61-9aec-83519df0923c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:04.291340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 501/1017] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:04.297946 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40028c46-fbb2-4d61-9aec-83519df0923c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:04.349321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f396c51-88ee-46dc-94a8-812a2ea458a1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:04.389334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:04.391259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeea24-c27c-4262-ad6f-56e0f0a37fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:04.392397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeea24-c27c-4262-ad6f-56e0f0a37fac None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:04.392823 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeea24-c27c-4262-ad6f-56e0f0a37fac None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:04.393103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeea24-c27c-4262-ad6f-56e0f0a37fac None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:04.393806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeea24-c27c-4262-ad6f-56e0f0a37fac None admin] RBAC: Adding query filter params (type=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:04.394162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeea24-c27c-4262-ad6f-56e0f0a37fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:04.395549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeea24-c27c-4262-ad6f-56e0f0a37fac None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:04.405433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 517/1018] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 13 08:49:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:04.407179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeea24-c27c-4262-ad6f-56e0f0a37fac None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:04.425645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40028c46-fbb2-4d61-9aec-83519df0923c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:04.455897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:04.464882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e678f3-81f7-4f3d-8163-1709e27fa3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:04.465537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e678f3-81f7-4f3d-8163-1709e27fa3b0 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:04.466120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e678f3-81f7-4f3d-8163-1709e27fa3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:04.466120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e678f3-81f7-4f3d-8163-1709e27fa3b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:04.466954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e678f3-81f7-4f3d-8163-1709e27fa3b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:04.468961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e678f3-81f7-4f3d-8163-1709e27fa3b0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:04.512554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeea24-c27c-4262-ad6f-56e0f0a37fac None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:04.547060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58ce62-f7e8-4eb6-b034-0d1129ae3ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:04.547665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58ce62-f7e8-4eb6-b034-0d1129ae3ab8 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:04.547946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58ce62-f7e8-4eb6-b034-0d1129ae3ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:04.548234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58ce62-f7e8-4eb6-b034-0d1129ae3ab8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:04.548947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58ce62-f7e8-4eb6-b034-0d1129ae3ab8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:04.549500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 502/1019] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:49:04 2022] POST /identity/v3/projects => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:49:04.550485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58ce62-f7e8-4eb6-b034-0d1129ae3ab8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:04.638071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 518/1020] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 13 08:49:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:49:05.630453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7c21-acf8-4564-8bce-d180a8ce62f4 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:05.630959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7c21-acf8-4564-8bce-d180a8ce62f4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:05.631037 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7c21-acf8-4564-8bce-d180a8ce62f4 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:05.632494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 503/1021] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:05.637352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c6f41-a42c-45c0-9f94-d8ec9f25cbe6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:05.637771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c6f41-a42c-45c0-9f94-d8ec9f25cbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:05.637949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c6f41-a42c-45c0-9f94-d8ec9f25cbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:05.655511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-615c6f41-a42c-45c0-9f94-d8ec9f25cbe6 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:05.656644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-615c6f41-a42c-45c0-9f94-d8ec9f25cbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:05.732858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 519/1022] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:05.755250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d7c21-acf8-4564-8bce-d180a8ce62f4 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:05.860241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d7c21-acf8-4564-8bce-d180a8ce62f4 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:05.884386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:05.896219 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc528d9-9750-47cc-8098-3293c8a37162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:05.904678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc528d9-9750-47cc-8098-3293c8a37162 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:05.904678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc528d9-9750-47cc-8098-3293c8a37162 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:05.904678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc528d9-9750-47cc-8098-3293c8a37162 None admin] PATH_INFO: `/v3/services/image` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:05.904678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc528d9-9750-47cc-8098-3293c8a37162 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:05.904678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc528d9-9750-47cc-8098-3293c8a37162 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3fc528d9-9750-47cc-8098-3293c8a37162 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:05.911624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:05.912572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 08:49:05.913415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:05.920114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 504/1023] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Jul 13 08:49:05 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:05.920245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c6f41-a42c-45c0-9f94-d8ec9f25cbe6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:05.946758 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:06.020929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c6f41-a42c-45c0-9f94-d8ec9f25cbe6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:06.054916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a282b0f-c02f-4c4e-9276-c081e1f660b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:06.055572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a282b0f-c02f-4c4e-9276-c081e1f660b2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:06.055848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a282b0f-c02f-4c4e-9276-c081e1f660b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:06.056119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a282b0f-c02f-4c4e-9276-c081e1f660b2 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:06.056812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a282b0f-c02f-4c4e-9276-c081e1f660b2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:06.057113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a282b0f-c02f-4c4e-9276-c081e1f660b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:06.058505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a282b0f-c02f-4c4e-9276-c081e1f660b2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:06.068194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 520/1024] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:06.069501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc528d9-9750-47cc-8098-3293c8a37162 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:06.173822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc528d9-9750-47cc-8098-3293c8a37162 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:06.208174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1198d-7891-4e5d-a221-60bb6b2f8ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:06.208835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1198d-7891-4e5d-a221-60bb6b2f8ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:06.209122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1198d-7891-4e5d-a221-60bb6b2f8ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:06.209397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1198d-7891-4e5d-a221-60bb6b2f8ec3 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:06.210126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1198d-7891-4e5d-a221-60bb6b2f8ec3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:06.210434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1198d-7891-4e5d-a221-60bb6b2f8ec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:06.211789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1198d-7891-4e5d-a221-60bb6b2f8ec3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:06.222018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 505/1025] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:06.223302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a282b0f-c02f-4c4e-9276-c081e1f660b2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:06.313310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:06.325746 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a282b0f-c02f-4c4e-9276-c081e1f660b2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:06.361848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25a9e4-7063-4b48-8b95-1be4a8e35616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:06.362541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25a9e4-7063-4b48-8b95-1be4a8e35616 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:06.362819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25a9e4-7063-4b48-8b95-1be4a8e35616 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:06.363140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25a9e4-7063-4b48-8b95-1be4a8e35616 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:06.363917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25a9e4-7063-4b48-8b95-1be4a8e35616 None admin] RBAC: Adding query filter params (interface=public, service_id=277c7a3ef9d14b92a096be4424ce09fc, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:06.364276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25a9e4-7063-4b48-8b95-1be4a8e35616 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:06.365763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25a9e4-7063-4b48-8b95-1be4a8e35616 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:06.376003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 521/1026] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 13 08:49:06 2022] GET /identity/v3/endpoints?service_id=277c7a3ef9d14b92a096be4424ce09fc&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:06.377198 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1198d-7891-4e5d-a221-60bb6b2f8ec3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:06.431897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:06.479552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1198d-7891-4e5d-a221-60bb6b2f8ec3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:06.513490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb3288-f6e8-4b86-a674-848ed85005ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:06.514174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb3288-f6e8-4b86-a674-848ed85005ef None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:06.514461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb3288-f6e8-4b86-a674-848ed85005ef None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:06.514729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb3288-f6e8-4b86-a674-848ed85005ef None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:06.516005 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb3288-f6e8-4b86-a674-848ed85005ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:06.517443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb3288-f6e8-4b86-a674-848ed85005ef None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:06.530298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 506/1027] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 13 08:49:06 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 08:49:06.640647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beed764-90fb-40e4-b666-7e6dbd73f1c1 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:06.641173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beed764-90fb-40e4-b666-7e6dbd73f1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:06.641406 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beed764-90fb-40e4-b666-7e6dbd73f1c1 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:06.643109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 522/1028] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:06.648572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c8ea4-b45a-45bd-94d4-cf34abd70947 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:06.649105 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c8ea4-b45a-45bd-94d4-cf34abd70947 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:06.649105 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c8ea4-b45a-45bd-94d4-cf34abd70947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:06.668449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-928c8ea4-b45a-45bd-94d4-cf34abd70947 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:06.669878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-928c8ea4-b45a-45bd-94d4-cf34abd70947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:06.743286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 507/1029] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:06.768257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beed764-90fb-40e4-b666-7e6dbd73f1c1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:06.799114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31dea4-1897-465a-b251-21b14f723953 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:06.799427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31dea4-1897-465a-b251-21b14f723953 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:06.799664 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31dea4-1897-465a-b251-21b14f723953 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:06.801218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 508/1030] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:06.806506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da6413-64b5-4e0d-a843-975348073ab1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:06.806795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da6413-64b5-4e0d-a843-975348073ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:06.807078 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da6413-64b5-4e0d-a843-975348073ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:06.823946 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c7da6413-64b5-4e0d-a843-975348073ab1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:06.825127 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c7da6413-64b5-4e0d-a843-975348073ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:06.872699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:06.875282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beed764-90fb-40e4-b666-7e6dbd73f1c1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:06.900493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 509/1031] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:49:06.914227 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891dd43-b180-4a48-9c23-8a0fff55ac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:06.915017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891dd43-b180-4a48-9c23-8a0fff55ac40 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:06.915295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891dd43-b180-4a48-9c23-8a0fff55ac40 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:06.915566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891dd43-b180-4a48-9c23-8a0fff55ac40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:06.916710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891dd43-b180-4a48-9c23-8a0fff55ac40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:06.918158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891dd43-b180-4a48-9c23-8a0fff55ac40 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:06.922073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 523/1032] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:06.925037 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da6413-64b5-4e0d-a843-975348073ab1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:06.925396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891dd43-b180-4a48-9c23-8a0fff55ac40 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:06.962397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:07.040309 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da6413-64b5-4e0d-a843-975348073ab1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:07.047437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891dd43-b180-4a48-9c23-8a0fff55ac40 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:07.080016 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c178cf-6167-460e-9c7a-0f53787b3f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:07.080827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c178cf-6167-460e-9c7a-0f53787b3f55 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:07.081109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c178cf-6167-460e-9c7a-0f53787b3f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:07.081383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c178cf-6167-460e-9c7a-0f53787b3f55 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:07.083783 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80159e0b-9bb4-4c8c-a826-05f46ae421f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:07.084453 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80159e0b-9bb4-4c8c-a826-05f46ae421f5 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:07.088860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80159e0b-9bb4-4c8c-a826-05f46ae421f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:07.088860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80159e0b-9bb4-4c8c-a826-05f46ae421f5 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:07.088860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80159e0b-9bb4-4c8c-a826-05f46ae421f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:07.088860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80159e0b-9bb4-4c8c-a826-05f46ae421f5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:07.092264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c178cf-6167-460e-9c7a-0f53787b3f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-89c178cf-6167-460e-9c7a-0f53787b3f55 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:07.100845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:07.101860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:07.102852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 510/1033] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 13 08:49:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:07.107918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c178cf-6167-460e-9c7a-0f53787b3f55 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:07.143764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:07.206901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 524/1034] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:49:06 2022] POST /identity/v3/users => generated 285 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:49:07.212382 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c178cf-6167-460e-9c7a-0f53787b3f55 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:07.253636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a850737-4819-493b-9e91-d1eb5006ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:07.254355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a850737-4819-493b-9e91-d1eb5006ba67 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:07.254638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a850737-4819-493b-9e91-d1eb5006ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:07.254910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a850737-4819-493b-9e91-d1eb5006ba67 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:07.255680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a850737-4819-493b-9e91-d1eb5006ba67 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:07.255987 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a850737-4819-493b-9e91-d1eb5006ba67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:07.257392 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a850737-4819-493b-9e91-d1eb5006ba67 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:07.272206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 511/1035] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 13 08:49:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:07.274096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80159e0b-9bb4-4c8c-a826-05f46ae421f5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:07.361207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:07.384120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80159e0b-9bb4-4c8c-a826-05f46ae421f5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:07.422344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f81b87-cdc9-413d-9cb1-990735ace840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:07.423156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f81b87-cdc9-413d-9cb1-990735ace840 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:07.423441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f81b87-cdc9-413d-9cb1-990735ace840 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:07.423722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f81b87-cdc9-413d-9cb1-990735ace840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:07.436044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f81b87-cdc9-413d-9cb1-990735ace840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:07.437565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f81b87-cdc9-413d-9cb1-990735ace840 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e9f81b87-cdc9-413d-9cb1-990735ace840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:49:07.455580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:07.456604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:07.457521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:49:07.457521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:49:07.457521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:07.457521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:07.457521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 525/1036] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:07.462552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a850737-4819-493b-9e91-d1eb5006ba67 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:07.563657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:07.568012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a850737-4819-493b-9e91-d1eb5006ba67 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:07.605522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e951a43-0a38-47c4-920b-de252dcd94dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:07.606128 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e951a43-0a38-47c4-920b-de252dcd94dc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:07.606419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e951a43-0a38-47c4-920b-de252dcd94dc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:07.606695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e951a43-0a38-47c4-920b-de252dcd94dc None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:07.607414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e951a43-0a38-47c4-920b-de252dcd94dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:07.607723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e951a43-0a38-47c4-920b-de252dcd94dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:07.609594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e951a43-0a38-47c4-920b-de252dcd94dc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:07.621679 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 512/1037] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:07.628352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f81b87-cdc9-413d-9cb1-990735ace840 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:07.733379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f81b87-cdc9-413d-9cb1-990735ace840 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:07.760453 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:07.769279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7353e-49db-43fe-b686-7756ba0ab6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:07.770087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7353e-49db-43fe-b686-7756ba0ab6bd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:07.770368 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7353e-49db-43fe-b686-7756ba0ab6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:07.770638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7353e-49db-43fe-b686-7756ba0ab6bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:07.778395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7353e-49db-43fe-b686-7756ba0ab6bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:07.779818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7353e-49db-43fe-b686-7756ba0ab6bd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8cd7353e-49db-43fe-b686-7756ba0ab6bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:49:07.794540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:07.795716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:07.796942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:07.796942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:07.796942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:49:07.796942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:49:07.796942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:49:07.796942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:49:07.796942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:49:07.796942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:49:07.796942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:49:07.796942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:07.803800 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 526/1038] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 13 08:49:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:07.805081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e951a43-0a38-47c4-920b-de252dcd94dc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:07.907578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e951a43-0a38-47c4-920b-de252dcd94dc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:07.941267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68b318-ec19-4daa-a12a-5a17c696e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:07.941930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68b318-ec19-4daa-a12a-5a17c696e36a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:07.942244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68b318-ec19-4daa-a12a-5a17c696e36a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:07.942519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68b318-ec19-4daa-a12a-5a17c696e36a None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:07.943349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68b318-ec19-4daa-a12a-5a17c696e36a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:07.943658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68b318-ec19-4daa-a12a-5a17c696e36a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:07.945090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68b318-ec19-4daa-a12a-5a17c696e36a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:07.966504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 513/1039] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 13 08:49:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:07.968912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7353e-49db-43fe-b686-7756ba0ab6bd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:08.070583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7353e-49db-43fe-b686-7756ba0ab6bd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:08.104860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f88789-e12e-4790-90cd-4aeb8953bede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:08.105544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f88789-e12e-4790-90cd-4aeb8953bede None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:08.105820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f88789-e12e-4790-90cd-4aeb8953bede None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:08.106118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f88789-e12e-4790-90cd-4aeb8953bede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:08.161672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:08.174451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f88789-e12e-4790-90cd-4aeb8953bede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:08.175900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f88789-e12e-4790-90cd-4aeb8953bede None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:08.278185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 527/1040] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:49:08.279440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68b318-ec19-4daa-a12a-5a17c696e36a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:08.299098 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:08.387136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68b318-ec19-4daa-a12a-5a17c696e36a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:08.424415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d51dc-8189-48e5-a54d-cec49458d60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:08.425131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d51dc-8189-48e5-a54d-cec49458d60c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:08.425412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d51dc-8189-48e5-a54d-cec49458d60c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:08.425685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d51dc-8189-48e5-a54d-cec49458d60c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:08.436204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d51dc-8189-48e5-a54d-cec49458d60c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:08.438179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d51dc-8189-48e5-a54d-cec49458d60c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-e67d51dc-8189-48e5-a54d-cec49458d60c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:49:08.456789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:08.457715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:08.463334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:49:08.463334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:49:08.463334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:08.463334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:08.463334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 514/1041] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:08.465202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f88789-e12e-4790-90cd-4aeb8953bede None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:08.553391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:08.566641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f88789-e12e-4790-90cd-4aeb8953bede None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:08.586490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0bec1-18cb-47a4-8df6-a76b7c4a1c2b None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:08.586772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0bec1-18cb-47a4-8df6-a76b7c4a1c2b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:08.587058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0bec1-18cb-47a4-8df6-a76b7c4a1c2b None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:08.588413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 515/1042] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:49:08.593166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6ccd0-2c2b-49ee-beff-e21bf37d6331 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:08.593453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6ccd0-2c2b-49ee-beff-e21bf37d6331 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:08.593741 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6ccd0-2c2b-49ee-beff-e21bf37d6331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:08.603823 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef81a2-556a-4f50-856c-41cebe35701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:08.605207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef81a2-556a-4f50-856c-41cebe35701c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:08.605669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef81a2-556a-4f50-856c-41cebe35701c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:08.606181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef81a2-556a-4f50-856c-41cebe35701c None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:08.607348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef81a2-556a-4f50-856c-41cebe35701c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:08.607843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef81a2-556a-4f50-856c-41cebe35701c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:08.609578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef81a2-556a-4f50-856c-41cebe35701c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:08.610471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-58a6ccd0-2c2b-49ee-beff-e21bf37d6331 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:08.611648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-58a6ccd0-2c2b-49ee-beff-e21bf37d6331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:08.624293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 528/1043] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:08.630770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef81a2-556a-4f50-856c-41cebe35701c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:08.682749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:08.698053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 516/1044] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:49:08.713343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6ccd0-2c2b-49ee-beff-e21bf37d6331 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:08.733558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef81a2-556a-4f50-856c-41cebe35701c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:08.771065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f40be-c9df-4b2c-82a6-0974d3dcfd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:08.772350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f40be-c9df-4b2c-82a6-0974d3dcfd39 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:08.772782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f40be-c9df-4b2c-82a6-0974d3dcfd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:08.773063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f40be-c9df-4b2c-82a6-0974d3dcfd39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:08.780557 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f40be-c9df-4b2c-82a6-0974d3dcfd39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:08.782093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f40be-c9df-4b2c-82a6-0974d3dcfd39 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-231f40be-c9df-4b2c-82a6-0974d3dcfd39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:49:08.796938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:08.797942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:08.798898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:49:08.798898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:49:08.798898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:49:08.798898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:49:08.798898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:49:08.798898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:49:08.798898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:49:08.798898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:08.802284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 529/1045] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:49:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:08.803804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f40be-c9df-4b2c-82a6-0974d3dcfd39 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:08.816127 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:08.831525 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6ccd0-2c2b-49ee-beff-e21bf37d6331 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:08.869858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad583c9-1cd8-4ee8-a779-e0b528df4714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:08.870674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad583c9-1cd8-4ee8-a779-e0b528df4714 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:08.870955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad583c9-1cd8-4ee8-a779-e0b528df4714 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:08.871226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad583c9-1cd8-4ee8-a779-e0b528df4714 None admin] PATH_INFO: `/v3/services/image` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:08.872152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad583c9-1cd8-4ee8-a779-e0b528df4714 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:08.875293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad583c9-1cd8-4ee8-a779-e0b528df4714 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-8ad583c9-1cd8-4ee8-a779-e0b528df4714 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:08.888907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:08.889927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:08.890981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 517/1046] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Jul 13 08:49:08 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:08.896255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad583c9-1cd8-4ee8-a779-e0b528df4714 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:08.910887 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f40be-c9df-4b2c-82a6-0974d3dcfd39 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:08.947592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e0990-3820-44f6-9f05-f7fe22e80a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:08.948172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e0990-3820-44f6-9f05-f7fe22e80a60 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:08.948455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e0990-3820-44f6-9f05-f7fe22e80a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:08.948734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e0990-3820-44f6-9f05-f7fe22e80a60 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:08.949515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e0990-3820-44f6-9f05-f7fe22e80a60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:08.949823 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e0990-3820-44f6-9f05-f7fe22e80a60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:08.951334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e0990-3820-44f6-9f05-f7fe22e80a60 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:08.963535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 530/1047] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:49:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:08.969942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e0990-3820-44f6-9f05-f7fe22e80a60 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:08.974453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:09.018226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad583c9-1cd8-4ee8-a779-e0b528df4714 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:09.040532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:09.053945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151f065-8589-4261-b478-661954d46764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:09.054835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151f065-8589-4261-b478-661954d46764 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:09.055255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151f065-8589-4261-b478-661954d46764 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:09.055669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151f065-8589-4261-b478-661954d46764 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:09.056503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151f065-8589-4261-b478-661954d46764 None admin] RBAC: Adding query filter params (name=image) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:09.056967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151f065-8589-4261-b478-661954d46764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:09.058538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151f065-8589-4261-b478-661954d46764 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:09.065055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 518/1048] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:09.070474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151f065-8589-4261-b478-661954d46764 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:09.077394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e0990-3820-44f6-9f05-f7fe22e80a60 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:09.145812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01b102-48fe-4b38-9ef5-252b372e89ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:09.146491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01b102-48fe-4b38-9ef5-252b372e89ce None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:09.146777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01b102-48fe-4b38-9ef5-252b372e89ce None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:09.147049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01b102-48fe-4b38-9ef5-252b372e89ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:09.147870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01b102-48fe-4b38-9ef5-252b372e89ce None admin] RBAC: Adding query filter params (role.id=609bba755f3f46f1b91a203b0a8ea404, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=8af26e036f4e466ebb923771c1f5f502) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:09.148176 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01b102-48fe-4b38-9ef5-252b372e89ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:09.149599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01b102-48fe-4b38-9ef5-252b372e89ce None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:09.155366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 531/1049] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:49:08 2022] GET /identity/v3/role_assignments?user.id=8af26e036f4e466ebb923771c1f5f502&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=609bba755f3f46f1b91a203b0a8ea404 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:09.229684 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151f065-8589-4261-b478-661954d46764 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:09.259235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:09.268744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f94584-5762-421e-967e-02fb5c0b6292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:09.269430 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f94584-5762-421e-967e-02fb5c0b6292 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:09.269713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f94584-5762-421e-967e-02fb5c0b6292 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:09.269991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f94584-5762-421e-967e-02fb5c0b6292 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:09.270827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f94584-5762-421e-967e-02fb5c0b6292 None admin] RBAC: Adding query filter params (type=image) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:09.276478 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f94584-5762-421e-967e-02fb5c0b6292 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:09.276478 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f94584-5762-421e-967e-02fb5c0b6292 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:09.278377 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 519/1050] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:09.284315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01b102-48fe-4b38-9ef5-252b372e89ce None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:09.317564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c5a5d-4c85-4d77-915d-a0c406e5971c None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:09.318255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c5a5d-4c85-4d77-915d-a0c406e5971c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:09.318781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c5a5d-4c85-4d77-915d-a0c406e5971c None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:09.320263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 520/1051] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:49:09.326274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6460c49-ef4e-47ba-9edf-2cb1affb4306 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:09.326558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6460c49-ef4e-47ba-9edf-2cb1affb4306 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:09.326831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6460c49-ef4e-47ba-9edf-2cb1affb4306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:09.355713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a6460c49-ef4e-47ba-9edf-2cb1affb4306 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:09.356957 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a6460c49-ef4e-47ba-9edf-2cb1affb4306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:09.395010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01b102-48fe-4b38-9ef5-252b372e89ce None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:09.429921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:09.434929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07a811-fc20-4363-9b83-641dda9a1850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:09.435585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07a811-fc20-4363-9b83-641dda9a1850 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:09.435870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07a811-fc20-4363-9b83-641dda9a1850 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:09.436146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07a811-fc20-4363-9b83-641dda9a1850 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:09.437415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07a811-fc20-4363-9b83-641dda9a1850 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:09.442398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 521/1052] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:49:09.442519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07a811-fc20-4363-9b83-641dda9a1850 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:09.462280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6460c49-ef4e-47ba-9edf-2cb1affb4306 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:09.538155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 532/1053] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 13 08:49:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:49:09.574358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6460c49-ef4e-47ba-9edf-2cb1affb4306 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:09.597347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:09.621799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a9d55-a847-41df-ba6b-9fe3b9c67b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:09.622642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a9d55-a847-41df-ba6b-9fe3b9c67b0e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:09.622925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a9d55-a847-41df-ba6b-9fe3b9c67b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:09.623201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a9d55-a847-41df-ba6b-9fe3b9c67b0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:09.635174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a9d55-a847-41df-ba6b-9fe3b9c67b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-a08a9d55-a847-41df-ba6b-9fe3b9c67b0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:09.643635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:09.644629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:09.645476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:09.650090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 522/1054] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:49:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:09.651362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07a811-fc20-4363-9b83-641dda9a1850 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:09.762688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07a811-fc20-4363-9b83-641dda9a1850 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:09.798118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1079b80-2b9a-4061-836f-dc431ca5baf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:09.798707 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1079b80-2b9a-4061-836f-dc431ca5baf0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:09.798996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1079b80-2b9a-4061-836f-dc431ca5baf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:09.799263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1079b80-2b9a-4061-836f-dc431ca5baf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:09.800011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1079b80-2b9a-4061-836f-dc431ca5baf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:09.800321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1079b80-2b9a-4061-836f-dc431ca5baf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:09.801693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1079b80-2b9a-4061-836f-dc431ca5baf0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:09.810493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 533/1055] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:49:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:09.816968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a9d55-a847-41df-ba6b-9fe3b9c67b0e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:09.920169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a9d55-a847-41df-ba6b-9fe3b9c67b0e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:09.954907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2dc5d-2341-498f-b833-72487c2cc311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:09.955772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2dc5d-2341-498f-b833-72487c2cc311 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:09.956051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2dc5d-2341-498f-b833-72487c2cc311 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:09.956327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2dc5d-2341-498f-b833-72487c2cc311 None admin] PATH_INFO: `/v3/users/50239ee1aad54d778601415de80aedf2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:09.969718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2dc5d-2341-498f-b833-72487c2cc311 None admin] RBAC: Authorizing `identity:get_user(user_id=50239ee1aad54d778601415de80aedf2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:09.971193 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2dc5d-2341-498f-b833-72487c2cc311 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:09.977491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 523/1056] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:09 2022] GET /identity/v3/users/50239ee1aad54d778601415de80aedf2 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:09.978717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1079b80-2b9a-4061-836f-dc431ca5baf0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:10.030771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:10.082718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1079b80-2b9a-4061-836f-dc431ca5baf0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:10.117569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8847c-341d-4ba7-8549-06f1ec7e0cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:10.118508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8847c-341d-4ba7-8549-06f1ec7e0cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:10.118938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8847c-341d-4ba7-8549-06f1ec7e0cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:10.119434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8847c-341d-4ba7-8549-06f1ec7e0cc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:10.187987 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8847c-341d-4ba7-8549-06f1ec7e0cc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:10.189687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8847c-341d-4ba7-8549-06f1ec7e0cc6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:10.239751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:10.295125 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 534/1057] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:49:10.301679 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2dc5d-2341-498f-b833-72487c2cc311 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:10.341046 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:10.409132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2dc5d-2341-498f-b833-72487c2cc311 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:10.443110 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e548e-0a23-4403-9193-04f3603718fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:10.443963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e548e-0a23-4403-9193-04f3603718fa None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:10.444244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e548e-0a23-4403-9193-04f3603718fa None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:10.444515 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e548e-0a23-4403-9193-04f3603718fa None admin] PATH_INFO: `/v3/projects/c24d65ca0050497bb2b20adf85830184` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:10.445672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e548e-0a23-4403-9193-04f3603718fa None admin] RBAC: Authorizing `identity:get_project(project_id=c24d65ca0050497bb2b20adf85830184)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:10.447121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e548e-0a23-4403-9193-04f3603718fa None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:10.448989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 524/1058] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:10 2022] GET /identity/v3/projects/c24d65ca0050497bb2b20adf85830184 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:10.455072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8847c-341d-4ba7-8549-06f1ec7e0cc6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:10.555045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8847c-341d-4ba7-8549-06f1ec7e0cc6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:10.588610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec15f07-6bbc-4bba-a9cf-ae3af660c1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:10.589257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec15f07-6bbc-4bba-a9cf-ae3af660c1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:10.589539 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec15f07-6bbc-4bba-a9cf-ae3af660c1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:10.589808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec15f07-6bbc-4bba-a9cf-ae3af660c1e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:10.590708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec15f07-6bbc-4bba-a9cf-ae3af660c1e6 None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=c24d65ca0050497bb2b20adf85830184, user.id=50239ee1aad54d778601415de80aedf2) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:10.590961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec15f07-6bbc-4bba-a9cf-ae3af660c1e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:10.592371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec15f07-6bbc-4bba-a9cf-ae3af660c1e6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:10.606310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 535/1059] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:49:10 2022] GET /identity/v3/role_assignments?user.id=50239ee1aad54d778601415de80aedf2&scope.project.id=c24d65ca0050497bb2b20adf85830184&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:11.199027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43951c06-0e96-4cad-a64b-378dfdfd0055 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:11.199585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43951c06-0e96-4cad-a64b-378dfdfd0055 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:11.199585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43951c06-0e96-4cad-a64b-378dfdfd0055 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:11.200961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 525/1060] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:11.206842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80edf78-e458-464e-93d0-784c3e8ee754 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:11.207229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80edf78-e458-464e-93d0-784c3e8ee754 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:11.207398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80edf78-e458-464e-93d0-784c3e8ee754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:11.224698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b80edf78-e458-464e-93d0-784c3e8ee754 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:11.225883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b80edf78-e458-464e-93d0-784c3e8ee754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:11.235584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:11.295385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 536/1061] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:11.317798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43951c06-0e96-4cad-a64b-378dfdfd0055 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:11.428604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43951c06-0e96-4cad-a64b-378dfdfd0055 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:11.430142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:11.464446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fc4f8-0369-4590-b965-1fae1de45cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:11.465178 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fc4f8-0369-4590-b965-1fae1de45cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:11.465459 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fc4f8-0369-4590-b965-1fae1de45cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:11.465734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fc4f8-0369-4590-b965-1fae1de45cb1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:11.476073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fc4f8-0369-4590-b965-1fae1de45cb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f20fc4f8-0369-4590-b965-1fae1de45cb1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:11.484321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:11.485254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:49:11.486638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:11.487649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 526/1062] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 13 08:49:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:11.491633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80edf78-e458-464e-93d0-784c3e8ee754 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:11.593293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80edf78-e458-464e-93d0-784c3e8ee754 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:11.627298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b51cf-0adb-4dcb-a915-618754bb69a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:11.627955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b51cf-0adb-4dcb-a915-618754bb69a3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:11.628232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b51cf-0adb-4dcb-a915-618754bb69a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:11.628497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b51cf-0adb-4dcb-a915-618754bb69a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:11.629235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b51cf-0adb-4dcb-a915-618754bb69a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:11.629537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b51cf-0adb-4dcb-a915-618754bb69a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:11.630928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b51cf-0adb-4dcb-a915-618754bb69a3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:11.640803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 537/1063] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 13 08:49:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:11.640993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af427-0269-4c68-90b6-8a183c49dab4 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:11.640993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af427-0269-4c68-90b6-8a183c49dab4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:11.641436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af427-0269-4c68-90b6-8a183c49dab4 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:11.642960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 527/1064] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:49:11.645527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b51cf-0adb-4dcb-a915-618754bb69a3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:11.647669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d9676-7d66-401d-94b1-98486cd2c3eb None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:11.647954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d9676-7d66-401d-94b1-98486cd2c3eb None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:11.648235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d9676-7d66-401d-94b1-98486cd2c3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:11.655086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:11.665016 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c22d9676-7d66-401d-94b1-98486cd2c3eb None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:11.669543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c22d9676-7d66-401d-94b1-98486cd2c3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:11.742149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 528/1065] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:49:11.753844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b51cf-0adb-4dcb-a915-618754bb69a3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:11.760715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d9676-7d66-401d-94b1-98486cd2c3eb None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:11.790211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1c7e7-6fff-496b-bb5a-c535b505c3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:11.791028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1c7e7-6fff-496b-bb5a-c535b505c3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:11.791311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1c7e7-6fff-496b-bb5a-c535b505c3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:11.791584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1c7e7-6fff-496b-bb5a-c535b505c3c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:11.803098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1c7e7-6fff-496b-bb5a-c535b505c3c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:11.804548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1c7e7-6fff-496b-bb5a-c535b505c3c7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-92c1c7e7-6fff-496b-bb5a-c535b505c3c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:49:11.822253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:11.823167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:11.824090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:49:11.824090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:49:11.824090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:11.824090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:11.828400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 538/1066] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:11.829048 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1c7e7-6fff-496b-bb5a-c535b505c3c7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:11.857253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:11.866703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d9676-7d66-401d-94b1-98486cd2c3eb None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:11.888927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:11.904604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed5593-9346-41bf-8816-61ee45bc7fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:11.905252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed5593-9346-41bf-8816-61ee45bc7fbb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:11.906127 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed5593-9346-41bf-8816-61ee45bc7fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:11.906414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed5593-9346-41bf-8816-61ee45bc7fbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:11.931483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1c7e7-6fff-496b-bb5a-c535b505c3c7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:11.968098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25512938-81d3-4ebf-bd6f-037cd2bd4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:11.968996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25512938-81d3-4ebf-bd6f-037cd2bd4b86 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:11.969475 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25512938-81d3-4ebf-bd6f-037cd2bd4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:11.969617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25512938-81d3-4ebf-bd6f-037cd2bd4b86 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:11.970308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25512938-81d3-4ebf-bd6f-037cd2bd4b86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:11.970614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25512938-81d3-4ebf-bd6f-037cd2bd4b86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:11.971977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25512938-81d3-4ebf-bd6f-037cd2bd4b86 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:11.976737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed5593-9346-41bf-8816-61ee45bc7fbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:11.978197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed5593-9346-41bf-8816-61ee45bc7fbb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:11.985838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 539/1067] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:11.991160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25512938-81d3-4ebf-bd6f-037cd2bd4b86 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:12.073542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:12.084944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 529/1068] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 08:49:12.091280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed5593-9346-41bf-8816-61ee45bc7fbb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:12.097573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25512938-81d3-4ebf-bd6f-037cd2bd4b86 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:12.120860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:12.136147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcc5dc-9943-4510-98e0-2ef14a99e208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:12.136728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc5dc-9943-4510-98e0-2ef14a99e208 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:12.137005 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc5dc-9943-4510-98e0-2ef14a99e208 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:12.137277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc5dc-9943-4510-98e0-2ef14a99e208 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:12.144931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcc5dc-9943-4510-98e0-2ef14a99e208 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:12.146318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcc5dc-9943-4510-98e0-2ef14a99e208 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-97bcc5dc-9943-4510-98e0-2ef14a99e208 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:49:12.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:12.162786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:12.163761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 540/1069] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 13 08:49:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:12.168923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcc5dc-9943-4510-98e0-2ef14a99e208 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:12.201175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed5593-9346-41bf-8816-61ee45bc7fbb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:12.236906 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3afe98-d9af-4446-a625-fbdca025fdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:12.237700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3afe98-d9af-4446-a625-fbdca025fdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:12.237980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3afe98-d9af-4446-a625-fbdca025fdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:12.238281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3afe98-d9af-4446-a625-fbdca025fdb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:12.248765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3afe98-d9af-4446-a625-fbdca025fdb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:12.250990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3afe98-d9af-4446-a625-fbdca025fdb7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-dd3afe98-d9af-4446-a625-fbdca025fdb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:49:12.269231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:12.270221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:12.271171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:49:12.271171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:49:12.271171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:12.271171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:12.271171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 530/1070] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:12.275628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcc5dc-9943-4510-98e0-2ef14a99e208 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:12.276149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3afe98-d9af-4446-a625-fbdca025fdb7 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:12.311016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3d0e-cb38-424b-b40e-93428a1820c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:12.312255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3d0e-cb38-424b-b40e-93428a1820c4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:12.312546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3d0e-cb38-424b-b40e-93428a1820c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:12.312853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3d0e-cb38-424b-b40e-93428a1820c4 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:12.313762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf3d0e-cb38-424b-b40e-93428a1820c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:12.314093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf3d0e-cb38-424b-b40e-93428a1820c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:12.315540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf3d0e-cb38-424b-b40e-93428a1820c4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:12.322940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:12.338330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 541/1071] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 13 08:49:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:12.344923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3d0e-cb38-424b-b40e-93428a1820c4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:12.381149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:12.391920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3afe98-d9af-4446-a625-fbdca025fdb7 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:12.431121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23e9b3-b32c-4001-a7bb-d1b75d4d01da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:12.431778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23e9b3-b32c-4001-a7bb-d1b75d4d01da None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:12.432061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23e9b3-b32c-4001-a7bb-d1b75d4d01da None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:12.432335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23e9b3-b32c-4001-a7bb-d1b75d4d01da None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:12.433052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23e9b3-b32c-4001-a7bb-d1b75d4d01da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:12.433355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23e9b3-b32c-4001-a7bb-d1b75d4d01da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:12.434747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23e9b3-b32c-4001-a7bb-d1b75d4d01da None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:12.447322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 531/1072] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:12.461813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3d0e-cb38-424b-b40e-93428a1820c4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:12.502128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55aa10-03f6-4fbf-852e-a93af372fc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:12.502884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55aa10-03f6-4fbf-852e-a93af372fc91 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:12.503165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55aa10-03f6-4fbf-852e-a93af372fc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:12.503439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55aa10-03f6-4fbf-852e-a93af372fc91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:12.515443 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55aa10-03f6-4fbf-852e-a93af372fc91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:12.517011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55aa10-03f6-4fbf-852e-a93af372fc91 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-fa55aa10-03f6-4fbf-852e-a93af372fc91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:49:12.536598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:12.537578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:12.544013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:49:12.544013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:49:12.544013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:12.544013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:12.544013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 542/1073] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:12.545565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23e9b3-b32c-4001-a7bb-d1b75d4d01da None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:12.624550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef572016-7bcb-470f-9ca7-10c0e2949d27 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:12.624964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef572016-7bcb-470f-9ca7-10c0e2949d27 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:12.625260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef572016-7bcb-470f-9ca7-10c0e2949d27 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:12.627370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 543/1074] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:49:12.632828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297460-729f-4acc-905e-e139eded7656 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:12.633108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297460-729f-4acc-905e-e139eded7656 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:12.633387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297460-729f-4acc-905e-e139eded7656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:12.653186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-33297460-729f-4acc-905e-e139eded7656 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:12.654469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-33297460-729f-4acc-905e-e139eded7656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:12.657723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23e9b3-b32c-4001-a7bb-d1b75d4d01da None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:12.660857 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:12.696979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec84c2-fedd-485d-91ed-9c1f2a992bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:12.696979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec84c2-fedd-485d-91ed-9c1f2a992bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:12.696979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec84c2-fedd-485d-91ed-9c1f2a992bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:12.696979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec84c2-fedd-485d-91ed-9c1f2a992bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:12.697444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec84c2-fedd-485d-91ed-9c1f2a992bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:12.697444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec84c2-fedd-485d-91ed-9c1f2a992bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:12.698344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec84c2-fedd-485d-91ed-9c1f2a992bd3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:12.711563 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 532/1075] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:12.717445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec84c2-fedd-485d-91ed-9c1f2a992bd3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:12.737367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 544/1076] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:12.759752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33297460-729f-4acc-905e-e139eded7656 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:12.779259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:12.826404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec84c2-fedd-485d-91ed-9c1f2a992bd3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:12.861315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ccf22-2776-4d68-b8d1-df64827388c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:12.862119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ccf22-2776-4d68-b8d1-df64827388c2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:12.862400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ccf22-2776-4d68-b8d1-df64827388c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:12.862670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ccf22-2776-4d68-b8d1-df64827388c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:12.864480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33297460-729f-4acc-905e-e139eded7656 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:12.869272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ccf22-2776-4d68-b8d1-df64827388c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:12.870737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ccf22-2776-4d68-b8d1-df64827388c2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-327ccf22-2776-4d68-b8d1-df64827388c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:49:12.884712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:12.885732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:12.887032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:49:12.887032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:49:12.887032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:49:12.887032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:49:12.887032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:49:12.887032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:49:12.887032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:49:12.887032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:12.887032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 533/1077] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:49:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:12.892208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ccf22-2776-4d68-b8d1-df64827388c2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:12.901148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85740630-f6b0-4fa8-be8a-e55cf30ec243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:12.901940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740630-f6b0-4fa8-be8a-e55cf30ec243 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:12.902255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740630-f6b0-4fa8-be8a-e55cf30ec243 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:12.902529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740630-f6b0-4fa8-be8a-e55cf30ec243 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:12.913638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85740630-f6b0-4fa8-be8a-e55cf30ec243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:12.916897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-85740630-f6b0-4fa8-be8a-e55cf30ec243 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:12.921753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:12.928283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 545/1078] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:49:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:12.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85740630-f6b0-4fa8-be8a-e55cf30ec243 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:12.991960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ccf22-2776-4d68-b8d1-df64827388c2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:13.005840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:13.025988 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8165ad-d600-4c78-9b75-e27042eb24e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:13.026662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8165ad-d600-4c78-9b75-e27042eb24e6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:13.026900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85740630-f6b0-4fa8-be8a-e55cf30ec243 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:13.026996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8165ad-d600-4c78-9b75-e27042eb24e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:13.027200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8165ad-d600-4c78-9b75-e27042eb24e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:13.027969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8165ad-d600-4c78-9b75-e27042eb24e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:13.028273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8165ad-d600-4c78-9b75-e27042eb24e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:13.029795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8165ad-d600-4c78-9b75-e27042eb24e6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:13.041294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 534/1079] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:49:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:13.047636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8165ad-d600-4c78-9b75-e27042eb24e6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:13.060711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07549e0-35fa-4d7d-b549-5dad174f7fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:13.065180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07549e0-35fa-4d7d-b549-5dad174f7fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:13.065180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07549e0-35fa-4d7d-b549-5dad174f7fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:13.065180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07549e0-35fa-4d7d-b549-5dad174f7fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:13.065180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07549e0-35fa-4d7d-b549-5dad174f7fd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:13.065180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07549e0-35fa-4d7d-b549-5dad174f7fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:13.065180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07549e0-35fa-4d7d-b549-5dad174f7fd0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:13.072742 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 546/1080] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:49:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:13.078949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07549e0-35fa-4d7d-b549-5dad174f7fd0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:13.170743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:13.185783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8165ad-d600-4c78-9b75-e27042eb24e6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:13.217562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07549e0-35fa-4d7d-b549-5dad174f7fd0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:13.220678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a12bf-4d4c-45fd-9e9b-9519cb4af0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:13.221560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a12bf-4d4c-45fd-9e9b-9519cb4af0d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:13.221837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a12bf-4d4c-45fd-9e9b-9519cb4af0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:13.222137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a12bf-4d4c-45fd-9e9b-9519cb4af0d8 None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/8af26e036f4e466ebb923771c1f5f502/roles/609bba755f3f46f1b91a203b0a8ea404` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:13.224418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a12bf-4d4c-45fd-9e9b-9519cb4af0d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=8af26e036f4e466ebb923771c1f5f502, role_id=609bba755f3f46f1b91a203b0a8ea404)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:13.225843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a12bf-4d4c-45fd-9e9b-9519cb4af0d8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:13.251877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732fa32-37e2-4f68-b1f3-9bcb3d46af31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:13.252741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732fa32-37e2-4f68-b1f3-9bcb3d46af31 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:13.253023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732fa32-37e2-4f68-b1f3-9bcb3d46af31 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:13.253298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732fa32-37e2-4f68-b1f3-9bcb3d46af31 None admin] PATH_INFO: `/v3/users/50239ee1aad54d778601415de80aedf2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:13.254513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732fa32-37e2-4f68-b1f3-9bcb3d46af31 None admin] RBAC: Authorizing `identity:get_user(user_id=50239ee1aad54d778601415de80aedf2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:13.255925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732fa32-37e2-4f68-b1f3-9bcb3d46af31 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:13.257535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 547/1081] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:13 2022] GET /identity/v3/users/50239ee1aad54d778601415de80aedf2 => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:13.263192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732fa32-37e2-4f68-b1f3-9bcb3d46af31 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:13.274889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:13.292705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 535/1082] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:49:13 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/8af26e036f4e466ebb923771c1f5f502/roles/609bba755f3f46f1b91a203b0a8ea404 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:49:13.371987 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732fa32-37e2-4f68-b1f3-9bcb3d46af31 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:13.407475 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071eb5a7-126c-4a8f-9204-f5a3a681cd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:13.408359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071eb5a7-126c-4a8f-9204-f5a3a681cd34 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:13.408639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071eb5a7-126c-4a8f-9204-f5a3a681cd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:13.408917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071eb5a7-126c-4a8f-9204-f5a3a681cd34 None admin] PATH_INFO: `/v3/projects/c24d65ca0050497bb2b20adf85830184` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:13.410258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071eb5a7-126c-4a8f-9204-f5a3a681cd34 None admin] RBAC: Authorizing `identity:get_project(project_id=c24d65ca0050497bb2b20adf85830184)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:13.411715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071eb5a7-126c-4a8f-9204-f5a3a681cd34 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:13.413440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 548/1083] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:13 2022] GET /identity/v3/projects/c24d65ca0050497bb2b20adf85830184 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:13.419740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a12bf-4d4c-45fd-9e9b-9519cb4af0d8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:13.521975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a12bf-4d4c-45fd-9e9b-9519cb4af0d8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:13.555791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531146e-a24c-43f3-a42a-d516058d1c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:13.556683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531146e-a24c-43f3-a42a-d516058d1c0c None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:13.556965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531146e-a24c-43f3-a42a-d516058d1c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:13.557241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531146e-a24c-43f3-a42a-d516058d1c0c None admin] PATH_INFO: `/v3/projects/c24d65ca0050497bb2b20adf85830184/users/50239ee1aad54d778601415de80aedf2/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:13.559204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531146e-a24c-43f3-a42a-d516058d1c0c None admin] RBAC: Authorizing `identity:create_grant(project_id=c24d65ca0050497bb2b20adf85830184, user_id=50239ee1aad54d778601415de80aedf2, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:13.560635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531146e-a24c-43f3-a42a-d516058d1c0c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:13.625384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 536/1084] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:49:13 2022] PUT /identity/v3/projects/c24d65ca0050497bb2b20adf85830184/users/50239ee1aad54d778601415de80aedf2/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:49:15.314784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a3eec-860e-409e-a93a-5c067180161d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:15.315319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a3eec-860e-409e-a93a-5c067180161d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:15.315319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a3eec-860e-409e-a93a-5c067180161d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:15.316699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 549/1085] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:15.321855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ae753-8d4a-423e-866c-fd7b1500af97 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:15.322263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ae753-8d4a-423e-866c-fd7b1500af97 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:15.322446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ae753-8d4a-423e-866c-fd7b1500af97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:15.329269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:15.344823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bf7ae753-8d4a-423e-866c-fd7b1500af97 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:15.346001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bf7ae753-8d4a-423e-866c-fd7b1500af97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:15.412866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 537/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:15.437113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a3eec-860e-409e-a93a-5c067180161d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:15.513563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:15.537919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a3eec-860e-409e-a93a-5c067180161d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:15.572964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b584e1e-be1f-497e-9bd3-8442e76cbb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:15.573702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584e1e-be1f-497e-9bd3-8442e76cbb69 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:15.573986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584e1e-be1f-497e-9bd3-8442e76cbb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:15.574293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584e1e-be1f-497e-9bd3-8442e76cbb69 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:15.585004 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b584e1e-be1f-497e-9bd3-8442e76cbb69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2b584e1e-be1f-497e-9bd3-8442e76cbb69 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:15.593316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:15.594459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:15.595379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 550/1087] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 13 08:49:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:15.600458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ae753-8d4a-423e-866c-fd7b1500af97 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:15.664702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4945895-8295-4375-8c39-0bdfba9b64fc None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:15.665087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4945895-8295-4375-8c39-0bdfba9b64fc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:15.665364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4945895-8295-4375-8c39-0bdfba9b64fc None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:15.666743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 551/1088] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 08:49:15.672628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4914b0-548b-4666-b384-8e75219c995d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:15.672905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4914b0-548b-4666-b384-8e75219c995d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:15.673178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4914b0-548b-4666-b384-8e75219c995d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:15.690615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bd4914b0-548b-4666-b384-8e75219c995d None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:15.691807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bd4914b0-548b-4666-b384-8e75219c995d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:15.698389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ae753-8d4a-423e-866c-fd7b1500af97 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:15.731057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a2acd-751a-491a-b69a-984798777e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:15.731710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a2acd-751a-491a-b69a-984798777e28 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:15.731994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a2acd-751a-491a-b69a-984798777e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:15.732265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a2acd-751a-491a-b69a-984798777e28 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:15.733001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a2acd-751a-491a-b69a-984798777e28 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:15.733297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a2acd-751a-491a-b69a-984798777e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:15.735542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a2acd-751a-491a-b69a-984798777e28 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:15.736846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:15.743941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 538/1089] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 13 08:49:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:15.750498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a2acd-751a-491a-b69a-984798777e28 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:15.758495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 552/1090] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:49:15.782233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4914b0-548b-4666-b384-8e75219c995d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:15.846335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:15.847773 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a2acd-751a-491a-b69a-984798777e28 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:15.880803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980c5ab-49b9-47ff-b898-783cf3975a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:15.881578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980c5ab-49b9-47ff-b898-783cf3975a38 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:15.881685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4914b0-548b-4666-b384-8e75219c995d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:15.881864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980c5ab-49b9-47ff-b898-783cf3975a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:15.882145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980c5ab-49b9-47ff-b898-783cf3975a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:15.892821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f980c5ab-49b9-47ff-b898-783cf3975a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:15.894294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f980c5ab-49b9-47ff-b898-783cf3975a38 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f980c5ab-49b9-47ff-b898-783cf3975a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:49:15.911717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:15.912675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:15.913592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:49:15.913592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:49:15.913592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:15.913592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:15.913592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 539/1091] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:15.918871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980c5ab-49b9-47ff-b898-783cf3975a38 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:15.918982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adb263-e4b8-4a70-bbac-5c465a7cef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:15.919691 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adb263-e4b8-4a70-bbac-5c465a7cef3d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:15.919967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adb263-e4b8-4a70-bbac-5c465a7cef3d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:15.920234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adb263-e4b8-4a70-bbac-5c465a7cef3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:15.930945 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9adb263-e4b8-4a70-bbac-5c465a7cef3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d9adb263-e4b8-4a70-bbac-5c465a7cef3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:15.938669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:15.939718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:15.941314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 553/1092] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:49:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:15.945791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adb263-e4b8-4a70-bbac-5c465a7cef3d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:15.970690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:16.020691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980c5ab-49b9-47ff-b898-783cf3975a38 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:16.044798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adb263-e4b8-4a70-bbac-5c465a7cef3d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:16.057420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65999fe5-a4fe-4b1c-bb3f-dd1b0e6d3b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:16.058100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65999fe5-a4fe-4b1c-bb3f-dd1b0e6d3b64 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:16.058378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65999fe5-a4fe-4b1c-bb3f-dd1b0e6d3b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:16.058657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65999fe5-a4fe-4b1c-bb3f-dd1b0e6d3b64 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:16.059360 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65999fe5-a4fe-4b1c-bb3f-dd1b0e6d3b64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:16.059657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65999fe5-a4fe-4b1c-bb3f-dd1b0e6d3b64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:16.061014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65999fe5-a4fe-4b1c-bb3f-dd1b0e6d3b64 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:16.070606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:16.073247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 540/1093] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:16.079062 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65999fe5-a4fe-4b1c-bb3f-dd1b0e6d3b64 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:16.084236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60384a32-4fa1-4833-b0d0-f731eda2e0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:16.084899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60384a32-4fa1-4833-b0d0-f731eda2e0ee None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:16.085231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60384a32-4fa1-4833-b0d0-f731eda2e0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:16.085551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60384a32-4fa1-4833-b0d0-f731eda2e0ee None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:16.086332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60384a32-4fa1-4833-b0d0-f731eda2e0ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:16.086642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60384a32-4fa1-4833-b0d0-f731eda2e0ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:16.088032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60384a32-4fa1-4833-b0d0-f731eda2e0ee None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:16.097435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 554/1094] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:49:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:16.103116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60384a32-4fa1-4833-b0d0-f731eda2e0ee None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:16.183015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65999fe5-a4fe-4b1c-bb3f-dd1b0e6d3b64 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:16.192326 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:16.201343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60384a32-4fa1-4833-b0d0-f731eda2e0ee None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:16.215977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77c9b3-ca32-40fb-8317-01b7863b4c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:16.216760 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77c9b3-ca32-40fb-8317-01b7863b4c0b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:16.217041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77c9b3-ca32-40fb-8317-01b7863b4c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:16.217320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77c9b3-ca32-40fb-8317-01b7863b4c0b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:16.224893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77c9b3-ca32-40fb-8317-01b7863b4c0b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:16.226359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77c9b3-ca32-40fb-8317-01b7863b4c0b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:16.236065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58084a8a-01a9-4420-86eb-ba012ff4d056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:16.236929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58084a8a-01a9-4420-86eb-ba012ff4d056 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:16.237214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58084a8a-01a9-4420-86eb-ba012ff4d056 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:16.237489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58084a8a-01a9-4420-86eb-ba012ff4d056 None admin] PATH_INFO: `/v3/users/50239ee1aad54d778601415de80aedf2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:16.238697 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58084a8a-01a9-4420-86eb-ba012ff4d056 None admin] RBAC: Authorizing `identity:get_user(user_id=50239ee1aad54d778601415de80aedf2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:16.240117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58084a8a-01a9-4420-86eb-ba012ff4d056 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-5b77c9b3-ca32-40fb-8317-01b7863b4c0b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:49:16.241484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:16.247456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:16.248505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:16.248505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:16.248505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:49:16.248505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:49:16.248505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:49:16.248505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:49:16.248505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:49:16.248505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:49:16.248505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 08:49:16.248505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:16.249545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 555/1095] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:16 2022] GET /identity/v3/users/50239ee1aad54d778601415de80aedf2 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:16.249620 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 541/1096] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 13 08:49:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:16.249696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58084a8a-01a9-4420-86eb-ba012ff4d056 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:16.249775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77c9b3-ca32-40fb-8317-01b7863b4c0b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:16.304195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:16.349652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58084a8a-01a9-4420-86eb-ba012ff4d056 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:16.353074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77c9b3-ca32-40fb-8317-01b7863b4c0b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:16.384477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2c26c-e279-435f-84ee-65662e19ea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:16.385118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2c26c-e279-435f-84ee-65662e19ea51 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:16.385403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2c26c-e279-435f-84ee-65662e19ea51 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:16.385681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2c26c-e279-435f-84ee-65662e19ea51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:16.386604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06066c-00d9-4ecf-a686-bfa91883c703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:16.387265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06066c-00d9-4ecf-a686-bfa91883c703 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:16.387543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06066c-00d9-4ecf-a686-bfa91883c703 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:16.387815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06066c-00d9-4ecf-a686-bfa91883c703 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:16.388636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06066c-00d9-4ecf-a686-bfa91883c703 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:16.388936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06066c-00d9-4ecf-a686-bfa91883c703 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:16.391322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06066c-00d9-4ecf-a686-bfa91883c703 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:16.409456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 542/1097] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 13 08:49:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:16.415162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06066c-00d9-4ecf-a686-bfa91883c703 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:16.441488 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:16.453775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2c26c-e279-435f-84ee-65662e19ea51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:16.455264 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2c26c-e279-435f-84ee-65662e19ea51 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:16.510251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:16.512583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06066c-00d9-4ecf-a686-bfa91883c703 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:16.545307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4077393-42b8-4cb3-9a52-0c3c837b291e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:16.546524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4077393-42b8-4cb3-9a52-0c3c837b291e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:16.546902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4077393-42b8-4cb3-9a52-0c3c837b291e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:16.547255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4077393-42b8-4cb3-9a52-0c3c837b291e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:16.555959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 556/1098] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:49:16.562427 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2c26c-e279-435f-84ee-65662e19ea51 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:16.613629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4077393-42b8-4cb3-9a52-0c3c837b291e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:16.615107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4077393-42b8-4cb3-9a52-0c3c837b291e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:16.641064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:16.659605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2c26c-e279-435f-84ee-65662e19ea51 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:16.693109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264d8f7-72ce-4851-af66-f339b325d4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:16.693955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264d8f7-72ce-4851-af66-f339b325d4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:16.694261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264d8f7-72ce-4851-af66-f339b325d4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:16.694536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264d8f7-72ce-4851-af66-f339b325d4b4 None admin] PATH_INFO: `/v3/projects/c24d65ca0050497bb2b20adf85830184` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:16.695687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264d8f7-72ce-4851-af66-f339b325d4b4 None admin] RBAC: Authorizing `identity:get_project(project_id=c24d65ca0050497bb2b20adf85830184)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:16.697108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264d8f7-72ce-4851-af66-f339b325d4b4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:16.698813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 557/1099] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:16 2022] GET /identity/v3/projects/c24d65ca0050497bb2b20adf85830184 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:16.704637 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264d8f7-72ce-4851-af66-f339b325d4b4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:16.713161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 543/1100] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:49:16.717608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4077393-42b8-4cb3-9a52-0c3c837b291e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:16.733203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:16.809306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264d8f7-72ce-4851-af66-f339b325d4b4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:16.816090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4077393-42b8-4cb3-9a52-0c3c837b291e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:16.841749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:16.844156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5bdf1-8d8c-4b3b-a25d-611d82aa6f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:16.844823 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5bdf1-8d8c-4b3b-a25d-611d82aa6f60 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:16.845102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5bdf1-8d8c-4b3b-a25d-611d82aa6f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:16.845369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5bdf1-8d8c-4b3b-a25d-611d82aa6f60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:16.846228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5bdf1-8d8c-4b3b-a25d-611d82aa6f60 None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=c24d65ca0050497bb2b20adf85830184, user.id=50239ee1aad54d778601415de80aedf2) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:16.846538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5bdf1-8d8c-4b3b-a25d-611d82aa6f60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:16.847953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5bdf1-8d8c-4b3b-a25d-611d82aa6f60 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:16.850374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a01d0f-b274-45dc-af0a-f9bdf982eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:16.851155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a01d0f-b274-45dc-af0a-f9bdf982eed8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:16.851436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a01d0f-b274-45dc-af0a-f9bdf982eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:16.851931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a01d0f-b274-45dc-af0a-f9bdf982eed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:16.853672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 558/1101] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:49:16 2022] GET /identity/v3/role_assignments?user.id=50239ee1aad54d778601415de80aedf2&scope.project.id=c24d65ca0050497bb2b20adf85830184&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:16.863296 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a01d0f-b274-45dc-af0a-f9bdf982eed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:16.864909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a01d0f-b274-45dc-af0a-f9bdf982eed8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-90a01d0f-b274-45dc-af0a-f9bdf982eed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:49:16.884958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:16.886131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:16.887051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:49:16.887051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:49:16.887051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:49:16.887051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:16.892064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 544/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:16.893045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5bdf1-8d8c-4b3b-a25d-611d82aa6f60 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:16.986328 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:16.998295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5bdf1-8d8c-4b3b-a25d-611d82aa6f60 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:17.033527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a2b4c-6be8-4ffe-b0d9-2873961506a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:17.034248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a2b4c-6be8-4ffe-b0d9-2873961506a8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:17.034529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a2b4c-6be8-4ffe-b0d9-2873961506a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:17.034800 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a2b4c-6be8-4ffe-b0d9-2873961506a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:17.035511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a2b4c-6be8-4ffe-b0d9-2873961506a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:17.035813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a2b4c-6be8-4ffe-b0d9-2873961506a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:17.037173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a2b4c-6be8-4ffe-b0d9-2873961506a8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:17.049446 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 559/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:17.055130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a01d0f-b274-45dc-af0a-f9bdf982eed8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:17.155196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a01d0f-b274-45dc-af0a-f9bdf982eed8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:17.188935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d48ac-cded-4b79-8375-5756b0876c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:17.189721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d48ac-cded-4b79-8375-5756b0876c62 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:17.190000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d48ac-cded-4b79-8375-5756b0876c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:17.190299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d48ac-cded-4b79-8375-5756b0876c62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:17.196995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d48ac-cded-4b79-8375-5756b0876c62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:17.198583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d48ac-cded-4b79-8375-5756b0876c62 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-056d48ac-cded-4b79-8375-5756b0876c62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:49:17.212871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:17.213878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:17.214812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:49:17.214812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:49:17.214812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:49:17.214812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:49:17.214812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:49:17.214812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:49:17.214812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:49:17.214812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:17.214812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 545/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:49:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:17.220295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a2b4c-6be8-4ffe-b0d9-2873961506a8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:17.320511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a2b4c-6be8-4ffe-b0d9-2873961506a8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:17.358087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d1eea-4993-4846-a5e5-daf1702402b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:17.359940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d1eea-4993-4846-a5e5-daf1702402b4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:17.360220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d1eea-4993-4846-a5e5-daf1702402b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:17.360493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d1eea-4993-4846-a5e5-daf1702402b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:17.361274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d1eea-4993-4846-a5e5-daf1702402b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:17.361580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d1eea-4993-4846-a5e5-daf1702402b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:17.363080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d1eea-4993-4846-a5e5-daf1702402b4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:17.376296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 560/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:49:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:17.382191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d48ac-cded-4b79-8375-5756b0876c62 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:17.423840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:17.482063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d48ac-cded-4b79-8375-5756b0876c62 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:17.515042 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02738765-c42d-4ccf-b2a6-ac3fcb3bbe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:17.515832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02738765-c42d-4ccf-b2a6-ac3fcb3bbe3e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:17.516112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02738765-c42d-4ccf-b2a6-ac3fcb3bbe3e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:17.516386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02738765-c42d-4ccf-b2a6-ac3fcb3bbe3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:17.517201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02738765-c42d-4ccf-b2a6-ac3fcb3bbe3e None admin] RBAC: Adding query filter params (role.id=609bba755f3f46f1b91a203b0a8ea404, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=8af26e036f4e466ebb923771c1f5f502) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:17.517507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02738765-c42d-4ccf-b2a6-ac3fcb3bbe3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:17.518941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02738765-c42d-4ccf-b2a6-ac3fcb3bbe3e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:17.524796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 546/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:49:17 2022] GET /identity/v3/role_assignments?user.id=8af26e036f4e466ebb923771c1f5f502&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=609bba755f3f46f1b91a203b0a8ea404 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:18.924530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab505f75-59d1-489f-b8ab-cbc1e09f23b3 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:18.925088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab505f75-59d1-489f-b8ab-cbc1e09f23b3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:18.925088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab505f75-59d1-489f-b8ab-cbc1e09f23b3 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:18.926448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 561/1107] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:18.931501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2ad42-8195-4e7c-ac18-55caff468e36 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:18.931869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2ad42-8195-4e7c-ac18-55caff468e36 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:18.932070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2ad42-8195-4e7c-ac18-55caff468e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:18.948367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8cc2ad42-8195-4e7c-ac18-55caff468e36 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:18.949550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8cc2ad42-8195-4e7c-ac18-55caff468e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:19.014289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 547/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:19.039074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab505f75-59d1-489f-b8ab-cbc1e09f23b3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:19.050611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:19.137532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab505f75-59d1-489f-b8ab-cbc1e09f23b3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:19.170579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728c31d-35fc-4811-99cc-18ca132b4856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:19.172032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728c31d-35fc-4811-99cc-18ca132b4856 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:19.172319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728c31d-35fc-4811-99cc-18ca132b4856 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:19.172598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728c31d-35fc-4811-99cc-18ca132b4856 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:19.173797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728c31d-35fc-4811-99cc-18ca132b4856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:19.175325 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728c31d-35fc-4811-99cc-18ca132b4856 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:19.176885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 562/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:19.182508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2ad42-8195-4e7c-ac18-55caff468e36 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:19.211896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:19.278971 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2ad42-8195-4e7c-ac18-55caff468e36 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:19.312954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2757656-e8c8-4393-a5aa-8fbd6bf5c131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:19.313604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2757656-e8c8-4393-a5aa-8fbd6bf5c131 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:19.313888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2757656-e8c8-4393-a5aa-8fbd6bf5c131 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:19.314191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2757656-e8c8-4393-a5aa-8fbd6bf5c131 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:19.314904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2757656-e8c8-4393-a5aa-8fbd6bf5c131 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:19.316312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2757656-e8c8-4393-a5aa-8fbd6bf5c131 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:19.397988 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 548/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:49:19 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:49:21.449824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7ec20-0a23-4901-9504-447596012a6c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:21.450380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7ec20-0a23-4901-9504-447596012a6c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:21.450380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7ec20-0a23-4901-9504-447596012a6c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:21.451748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 563/1111] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:21.457505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b204a0-4fff-452b-a167-9a369ea2cab1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:21.457949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b204a0-4fff-452b-a167-9a369ea2cab1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:21.458088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b204a0-4fff-452b-a167-9a369ea2cab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:21.474650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-61b204a0-4fff-452b-a167-9a369ea2cab1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:21.475827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-61b204a0-4fff-452b-a167-9a369ea2cab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:21.541392 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 549/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:21.565429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7ec20-0a23-4901-9504-447596012a6c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:21.641576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:21.663959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7ec20-0a23-4901-9504-447596012a6c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:21.699332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ab04d-043d-4398-996e-f01b15e2daaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:21.700163 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ab04d-043d-4398-996e-f01b15e2daaf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:21.700439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ab04d-043d-4398-996e-f01b15e2daaf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:21.700706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ab04d-043d-4398-996e-f01b15e2daaf None admin] PATH_INFO: `/v3/roles/120eeffad7094e10a69166d19aaa6439` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:21.702728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ab04d-043d-4398-996e-f01b15e2daaf None admin] RBAC: Authorizing `identity:get_role(role_id=120eeffad7094e10a69166d19aaa6439)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:21.704097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ab04d-043d-4398-996e-f01b15e2daaf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:21.705433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 564/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:21 2022] GET /identity/v3/roles/120eeffad7094e10a69166d19aaa6439 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:21.711094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b204a0-4fff-452b-a167-9a369ea2cab1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:21.729017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:21.807564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b204a0-4fff-452b-a167-9a369ea2cab1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:21.840233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971c136-0f19-4b0a-b1de-71f237c32726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:21.841087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971c136-0f19-4b0a-b1de-71f237c32726 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:21.841367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971c136-0f19-4b0a-b1de-71f237c32726 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:21.841635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971c136-0f19-4b0a-b1de-71f237c32726 None admin] PATH_INFO: `/v3/users/9a88719267d44e069cd8f0b62e22c230` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:21.854135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971c136-0f19-4b0a-b1de-71f237c32726 None admin] RBAC: Authorizing `identity:get_user(user_id=9a88719267d44e069cd8f0b62e22c230)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:21.855568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971c136-0f19-4b0a-b1de-71f237c32726 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:21.857162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 550/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:21 2022] GET /identity/v3/users/9a88719267d44e069cd8f0b62e22c230 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:21.863521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ab04d-043d-4398-996e-f01b15e2daaf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:21.958658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ab04d-043d-4398-996e-f01b15e2daaf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:21.992118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcca8f-8529-4edd-a6bb-5e3b9263c4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:21.992746 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcca8f-8529-4edd-a6bb-5e3b9263c4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:21.993030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcca8f-8529-4edd-a6bb-5e3b9263c4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:21.993306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcca8f-8529-4edd-a6bb-5e3b9263c4b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:22.007088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:22.058376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcca8f-8529-4edd-a6bb-5e3b9263c4b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:22.059846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcca8f-8529-4edd-a6bb-5e3b9263c4b1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:22.154811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 565/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:49:22.161260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971c136-0f19-4b0a-b1de-71f237c32726 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:22.236718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:22.258859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971c136-0f19-4b0a-b1de-71f237c32726 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:22.410504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd795cfc-42da-4672-b7cd-99f087533d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:22.411357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd795cfc-42da-4672-b7cd-99f087533d78 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:22.411639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd795cfc-42da-4672-b7cd-99f087533d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:22.411913 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd795cfc-42da-4672-b7cd-99f087533d78 None admin] PATH_INFO: `/v3/projects/c24d65ca0050497bb2b20adf85830184` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:22.413074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd795cfc-42da-4672-b7cd-99f087533d78 None admin] RBAC: Authorizing `identity:get_project(project_id=c24d65ca0050497bb2b20adf85830184)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:22.414607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd795cfc-42da-4672-b7cd-99f087533d78 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:22.416283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 551/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:22 2022] GET /identity/v3/projects/c24d65ca0050497bb2b20adf85830184 => generated 311 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:22.423430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcca8f-8529-4edd-a6bb-5e3b9263c4b1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:22.467819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:22.520517 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcca8f-8529-4edd-a6bb-5e3b9263c4b1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:22.553741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41ad07-dc8d-4f60-88e2-df3cd8cc6265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:22.554407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41ad07-dc8d-4f60-88e2-df3cd8cc6265 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:22.554693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41ad07-dc8d-4f60-88e2-df3cd8cc6265 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:22.554968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41ad07-dc8d-4f60-88e2-df3cd8cc6265 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:22.555787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41ad07-dc8d-4f60-88e2-df3cd8cc6265 None admin] RBAC: Adding query filter params (role.id=120eeffad7094e10a69166d19aaa6439, scope.project.id=c24d65ca0050497bb2b20adf85830184, user.id=9a88719267d44e069cd8f0b62e22c230) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:22.556096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41ad07-dc8d-4f60-88e2-df3cd8cc6265 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:22.557518 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41ad07-dc8d-4f60-88e2-df3cd8cc6265 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:22.563558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 566/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:49:22 2022] GET /identity/v3/role_assignments?user.id=9a88719267d44e069cd8f0b62e22c230&scope.project.id=c24d65ca0050497bb2b20adf85830184&role.id=120eeffad7094e10a69166d19aaa6439 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:24.533770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c8545-0535-4640-b085-2b1ae69f4509 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:24.534375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c8545-0535-4640-b085-2b1ae69f4509 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:24.534375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c8545-0535-4640-b085-2b1ae69f4509 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:24.535669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 552/1118] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:24.540795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d61622-110a-4089-8d2a-4ccb1b1165f4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:24.543614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d61622-110a-4089-8d2a-4ccb1b1165f4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:24.543614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d61622-110a-4089-8d2a-4ccb1b1165f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:24.557986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-04d61622-110a-4089-8d2a-4ccb1b1165f4 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:24.559176 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-04d61622-110a-4089-8d2a-4ccb1b1165f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:24.622200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 567/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:24.644707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c8545-0535-4640-b085-2b1ae69f4509 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:24.743065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c8545-0535-4640-b085-2b1ae69f4509 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:24.775650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bff47-ef00-426c-a966-48a6378fb87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:24.776488 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bff47-ef00-426c-a966-48a6378fb87f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:24.776767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bff47-ef00-426c-a966-48a6378fb87f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:24.777037 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bff47-ef00-426c-a966-48a6378fb87f None admin] PATH_INFO: `/v3/roles/120eeffad7094e10a69166d19aaa6439` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:24.778526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bff47-ef00-426c-a966-48a6378fb87f None admin] RBAC: Authorizing `identity:get_role(role_id=120eeffad7094e10a69166d19aaa6439)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:24.780410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bff47-ef00-426c-a966-48a6378fb87f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:24.781902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 553/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:24 2022] GET /identity/v3/roles/120eeffad7094e10a69166d19aaa6439 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:24.787661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d61622-110a-4089-8d2a-4ccb1b1165f4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:24.825313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:24.884500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d61622-110a-4089-8d2a-4ccb1b1165f4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:24.917726 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41a8ac-621c-48a4-8de1-193e557ca13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:24.918856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41a8ac-621c-48a4-8de1-193e557ca13b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:24.919132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41a8ac-621c-48a4-8de1-193e557ca13b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:24.919398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41a8ac-621c-48a4-8de1-193e557ca13b None admin] PATH_INFO: `/v3/users/9a88719267d44e069cd8f0b62e22c230` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:24.920753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41a8ac-621c-48a4-8de1-193e557ca13b None admin] RBAC: Authorizing `identity:get_user(user_id=9a88719267d44e069cd8f0b62e22c230)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:24.922430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41a8ac-621c-48a4-8de1-193e557ca13b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:24.924015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 568/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:24 2022] GET /identity/v3/users/9a88719267d44e069cd8f0b62e22c230 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:24.929791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bff47-ef00-426c-a966-48a6378fb87f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:24.944930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:25.028819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bff47-ef00-426c-a966-48a6378fb87f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:25.061861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c5d2a-7493-4b89-96a5-e84c797a693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:25.062724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c5d2a-7493-4b89-96a5-e84c797a693e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:25.063008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c5d2a-7493-4b89-96a5-e84c797a693e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:25.063283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c5d2a-7493-4b89-96a5-e84c797a693e None admin] PATH_INFO: `/v3/projects/c24d65ca0050497bb2b20adf85830184` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:25.064447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c5d2a-7493-4b89-96a5-e84c797a693e None admin] RBAC: Authorizing `identity:get_project(project_id=c24d65ca0050497bb2b20adf85830184)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:25.065858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c5d2a-7493-4b89-96a5-e84c797a693e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:25.067540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 554/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:24 2022] GET /identity/v3/projects/c24d65ca0050497bb2b20adf85830184 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:25.073534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41a8ac-621c-48a4-8de1-193e557ca13b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:25.169774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41a8ac-621c-48a4-8de1-193e557ca13b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:25.176722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:25.202651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ff0bc-261b-4900-b524-12ed78a98f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:25.203507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ff0bc-261b-4900-b524-12ed78a98f51 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:25.203800 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ff0bc-261b-4900-b524-12ed78a98f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:25.204045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ff0bc-261b-4900-b524-12ed78a98f51 None admin] PATH_INFO: `/v3/projects/c24d65ca0050497bb2b20adf85830184/users/9a88719267d44e069cd8f0b62e22c230/roles/120eeffad7094e10a69166d19aaa6439` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:25.206225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ff0bc-261b-4900-b524-12ed78a98f51 None admin] RBAC: Authorizing `identity:create_grant(project_id=c24d65ca0050497bb2b20adf85830184, user_id=9a88719267d44e069cd8f0b62e22c230, role_id=120eeffad7094e10a69166d19aaa6439)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:25.207633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ff0bc-261b-4900-b524-12ed78a98f51 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:25.311736 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 569/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:49:25 2022] PUT /identity/v3/projects/c24d65ca0050497bb2b20adf85830184/users/9a88719267d44e069cd8f0b62e22c230/roles/120eeffad7094e10a69166d19aaa6439 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 08:49:27.301489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59a87-04cd-49e0-874e-a2e38628bd52 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:27.302024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59a87-04cd-49e0-874e-a2e38628bd52 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:27.302024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59a87-04cd-49e0-874e-a2e38628bd52 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:27.303414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 555/1124] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:27.308411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef1b52-d0db-43e3-97e4-864af27a1f25 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:27.308792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef1b52-d0db-43e3-97e4-864af27a1f25 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:27.308958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef1b52-d0db-43e3-97e4-864af27a1f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:27.325391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0fef1b52-d0db-43e3-97e4-864af27a1f25 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:27.327222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0fef1b52-d0db-43e3-97e4-864af27a1f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:27.391832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 570/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:27.415080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59a87-04cd-49e0-874e-a2e38628bd52 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:27.495048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:27.515656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59a87-04cd-49e0-874e-a2e38628bd52 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:27.550561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945cf83-3b02-447b-bcf1-39535e272df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:27.551394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945cf83-3b02-447b-bcf1-39535e272df0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:27.551720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945cf83-3b02-447b-bcf1-39535e272df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:27.551940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945cf83-3b02-447b-bcf1-39535e272df0 None admin] PATH_INFO: `/v3/roles/120eeffad7094e10a69166d19aaa6439` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:27.553368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945cf83-3b02-447b-bcf1-39535e272df0 None admin] RBAC: Authorizing `identity:get_role(role_id=120eeffad7094e10a69166d19aaa6439)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:27.554736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945cf83-3b02-447b-bcf1-39535e272df0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:27.556052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 556/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:27 2022] GET /identity/v3/roles/120eeffad7094e10a69166d19aaa6439 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:49:27.562448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef1b52-d0db-43e3-97e4-864af27a1f25 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:27.651930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:27.657484 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef1b52-d0db-43e3-97e4-864af27a1f25 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:27.691448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe054a9e-d66e-4f2c-a642-ed89467c3d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:27.692301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe054a9e-d66e-4f2c-a642-ed89467c3d4e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:27.692580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe054a9e-d66e-4f2c-a642-ed89467c3d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:27.692851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe054a9e-d66e-4f2c-a642-ed89467c3d4e None admin] PATH_INFO: `/v3/users/9a88719267d44e069cd8f0b62e22c230` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:27.694014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe054a9e-d66e-4f2c-a642-ed89467c3d4e None admin] RBAC: Authorizing `identity:get_user(user_id=9a88719267d44e069cd8f0b62e22c230)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:27.695470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe054a9e-d66e-4f2c-a642-ed89467c3d4e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:27.697123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 571/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:27 2022] GET /identity/v3/users/9a88719267d44e069cd8f0b62e22c230 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:27.702893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945cf83-3b02-447b-bcf1-39535e272df0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:27.798291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945cf83-3b02-447b-bcf1-39535e272df0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:27.830238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16628bf6-aed7-4d41-959e-328340299940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:27.830803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628bf6-aed7-4d41-959e-328340299940 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:27.831079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628bf6-aed7-4d41-959e-328340299940 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:27.831349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628bf6-aed7-4d41-959e-328340299940 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:27.847863 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:27.895828 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16628bf6-aed7-4d41-959e-328340299940 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:27.897246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16628bf6-aed7-4d41-959e-328340299940 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:27.989855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 557/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:49:27.995649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe054a9e-d66e-4f2c-a642-ed89467c3d4e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:28.092708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe054a9e-d66e-4f2c-a642-ed89467c3d4e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:28.125735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70471c4e-d445-45b2-bd3a-b96e90dee227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:28.126610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70471c4e-d445-45b2-bd3a-b96e90dee227 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:28.126894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70471c4e-d445-45b2-bd3a-b96e90dee227 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:28.127168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70471c4e-d445-45b2-bd3a-b96e90dee227 None admin] PATH_INFO: `/v3/projects/c24d65ca0050497bb2b20adf85830184` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:28.128324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70471c4e-d445-45b2-bd3a-b96e90dee227 None admin] RBAC: Authorizing `identity:get_project(project_id=c24d65ca0050497bb2b20adf85830184)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:28.129756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70471c4e-d445-45b2-bd3a-b96e90dee227 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:28.131460 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 572/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:27 2022] GET /identity/v3/projects/c24d65ca0050497bb2b20adf85830184 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:28.137502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16628bf6-aed7-4d41-959e-328340299940 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:28.184799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:28.234990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16628bf6-aed7-4d41-959e-328340299940 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:28.269148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54535d8a-9775-4dda-b065-7a68cb805f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:28.269789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54535d8a-9775-4dda-b065-7a68cb805f56 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:28.270093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54535d8a-9775-4dda-b065-7a68cb805f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:28.270369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54535d8a-9775-4dda-b065-7a68cb805f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:28.271187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54535d8a-9775-4dda-b065-7a68cb805f56 None admin] RBAC: Adding query filter params (role.id=120eeffad7094e10a69166d19aaa6439, scope.project.id=c24d65ca0050497bb2b20adf85830184, user.id=9a88719267d44e069cd8f0b62e22c230) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:28.271496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54535d8a-9775-4dda-b065-7a68cb805f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:28.272897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54535d8a-9775-4dda-b065-7a68cb805f56 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:28.278545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 558/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:49:28 2022] GET /identity/v3/role_assignments?user.id=9a88719267d44e069cd8f0b62e22c230&scope.project.id=c24d65ca0050497bb2b20adf85830184&role.id=120eeffad7094e10a69166d19aaa6439 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:30.307804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93614166-214c-44c1-80a4-6edec6f57c75 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:30.308376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93614166-214c-44c1-80a4-6edec6f57c75 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:30.308376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93614166-214c-44c1-80a4-6edec6f57c75 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:30.309724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 573/1131] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:30.315531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48248baa-efe7-439b-a8e9-d101772b1e68 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:30.315929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48248baa-efe7-439b-a8e9-d101772b1e68 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:30.316081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48248baa-efe7-439b-a8e9-d101772b1e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:30.332978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-48248baa-efe7-439b-a8e9-d101772b1e68 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:30.334253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-48248baa-efe7-439b-a8e9-d101772b1e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:30.398697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 559/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:30.422051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93614166-214c-44c1-80a4-6edec6f57c75 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:30.453301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:30.521113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93614166-214c-44c1-80a4-6edec6f57c75 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:30.553958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa4d3c-0763-4ec6-9ec2-5c7c1c77dbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:30.554764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa4d3c-0763-4ec6-9ec2-5c7c1c77dbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:30.555047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa4d3c-0763-4ec6-9ec2-5c7c1c77dbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:30.555324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa4d3c-0763-4ec6-9ec2-5c7c1c77dbf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:30.556479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa4d3c-0763-4ec6-9ec2-5c7c1c77dbf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:30.557963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa4d3c-0763-4ec6-9ec2-5c7c1c77dbf7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:30.559545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 574/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:30.565953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48248baa-efe7-439b-a8e9-d101772b1e68 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:30.605725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:30.660367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48248baa-efe7-439b-a8e9-d101772b1e68 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:30.694146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3013888-2343-4de2-8b31-95c188691221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:30.694790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3013888-2343-4de2-8b31-95c188691221 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:30.695074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3013888-2343-4de2-8b31-95c188691221 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:30.695350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3013888-2343-4de2-8b31-95c188691221 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:30.696056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3013888-2343-4de2-8b31-95c188691221 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:30.697469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3013888-2343-4de2-8b31-95c188691221 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:30.796367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 560/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:49:30 2022] POST /identity/v3/projects => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:49:32.840970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba8893-6a5b-4426-b9eb-d2f6bdd6a9b7 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:32.841528 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba8893-6a5b-4426-b9eb-d2f6bdd6a9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:32.841528 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba8893-6a5b-4426-b9eb-d2f6bdd6a9b7 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:32.842958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 575/1135] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:32.848135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277df4d-a01b-408e-9d50-00ab73e7b933 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:32.848578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277df4d-a01b-408e-9d50-00ab73e7b933 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:32.848688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277df4d-a01b-408e-9d50-00ab73e7b933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:32.865736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a277df4d-a01b-408e-9d50-00ab73e7b933 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:32.866955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a277df4d-a01b-408e-9d50-00ab73e7b933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:32.932943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 561/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:32.957660 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba8893-6a5b-4426-b9eb-d2f6bdd6a9b7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:33.054506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:33.058203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba8893-6a5b-4426-b9eb-d2f6bdd6a9b7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:33.093225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f5abd-a7e4-400d-a7f5-40c0aba086c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:33.094144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f5abd-a7e4-400d-a7f5-40c0aba086c0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:33.094479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f5abd-a7e4-400d-a7f5-40c0aba086c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:33.094945 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f5abd-a7e4-400d-a7f5-40c0aba086c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:33.096450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f5abd-a7e4-400d-a7f5-40c0aba086c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:33.097886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f5abd-a7e4-400d-a7f5-40c0aba086c0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:33.099588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 576/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:49:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:33.105250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277df4d-a01b-408e-9d50-00ab73e7b933 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:33.129315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:33.203465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277df4d-a01b-408e-9d50-00ab73e7b933 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:33.236730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3d189-7a3c-4907-9796-05fca5d9033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:33.237381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3d189-7a3c-4907-9796-05fca5d9033a None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:33.237665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3d189-7a3c-4907-9796-05fca5d9033a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:33.237947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3d189-7a3c-4907-9796-05fca5d9033a None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:33.238685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3d189-7a3c-4907-9796-05fca5d9033a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:33.240117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3d189-7a3c-4907-9796-05fca5d9033a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:33.322532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 562/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:49:33 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:49:35.386586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b77914-5704-425c-a1ef-c2854845f177 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:35.387041 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b77914-5704-425c-a1ef-c2854845f177 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:35.387134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b77914-5704-425c-a1ef-c2854845f177 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:35.388467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 577/1139] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:35.393797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9882ebd-f0e3-4fa2-b54a-379cd154bd27 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:35.394277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9882ebd-f0e3-4fa2-b54a-379cd154bd27 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:35.394369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9882ebd-f0e3-4fa2-b54a-379cd154bd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:35.411172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a9882ebd-f0e3-4fa2-b54a-379cd154bd27 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:35.412354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a9882ebd-f0e3-4fa2-b54a-379cd154bd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:35.476166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 563/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:35.499448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b77914-5704-425c-a1ef-c2854845f177 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:35.597774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b77914-5704-425c-a1ef-c2854845f177 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:35.630352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73914827-2c0f-48d7-a4d1-4f9da6fdb135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:35.631123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73914827-2c0f-48d7-a4d1-4f9da6fdb135 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:35.631403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73914827-2c0f-48d7-a4d1-4f9da6fdb135 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:35.631675 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73914827-2c0f-48d7-a4d1-4f9da6fdb135 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:35.641699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73914827-2c0f-48d7-a4d1-4f9da6fdb135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-73914827-2c0f-48d7-a4d1-4f9da6fdb135 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:35.649292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:35.650433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:35.651348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 578/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:49:35.657112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9882ebd-f0e3-4fa2-b54a-379cd154bd27 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:35.668897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:35.754993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9882ebd-f0e3-4fa2-b54a-379cd154bd27 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:35.787778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a5e0b-21d7-43e4-b1f7-eb53e78f7d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:35.788426 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a5e0b-21d7-43e4-b1f7-eb53e78f7d67 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:35.788713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a5e0b-21d7-43e4-b1f7-eb53e78f7d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:35.788990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a5e0b-21d7-43e4-b1f7-eb53e78f7d67 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:35.789727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a5e0b-21d7-43e4-b1f7-eb53e78f7d67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:35.790081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a5e0b-21d7-43e4-b1f7-eb53e78f7d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:35.791407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a5e0b-21d7-43e4-b1f7-eb53e78f7d67 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:35.800126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 564/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:35.805757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73914827-2c0f-48d7-a4d1-4f9da6fdb135 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:35.834460 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:35.903169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73914827-2c0f-48d7-a4d1-4f9da6fdb135 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:35.935428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7cee-e38f-4573-84a5-e0ed4ba1fa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:35.936820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7cee-e38f-4573-84a5-e0ed4ba1fa72 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:35.937095 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7cee-e38f-4573-84a5-e0ed4ba1fa72 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:35.937366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7cee-e38f-4573-84a5-e0ed4ba1fa72 None admin] PATH_INFO: `/v3/users/fd3fea5fa9b44faba6d2a276e51f0f65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:35.950589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7cee-e38f-4573-84a5-e0ed4ba1fa72 None admin] RBAC: Authorizing `identity:get_user(user_id=fd3fea5fa9b44faba6d2a276e51f0f65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:35.952338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7cee-e38f-4573-84a5-e0ed4ba1fa72 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:35.953981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 579/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:35 2022] GET /identity/v3/users/fd3fea5fa9b44faba6d2a276e51f0f65 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:35.959598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a5e0b-21d7-43e4-b1f7-eb53e78f7d67 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:36.027568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:36.056396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a5e0b-21d7-43e4-b1f7-eb53e78f7d67 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:36.090138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519833dc-580d-4170-9076-d92b1ffaacfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:36.090777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519833dc-580d-4170-9076-d92b1ffaacfc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:36.091047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519833dc-580d-4170-9076-d92b1ffaacfc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:36.091317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519833dc-580d-4170-9076-d92b1ffaacfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:36.160160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519833dc-580d-4170-9076-d92b1ffaacfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:36.161530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519833dc-580d-4170-9076-d92b1ffaacfc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:36.225967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:36.255887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 565/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:49:36.261677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7cee-e38f-4573-84a5-e0ed4ba1fa72 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:36.345986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:36.360360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7cee-e38f-4573-84a5-e0ed4ba1fa72 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:36.394096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279bb59-24d7-4cda-b4a3-108afe9807cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:36.394941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279bb59-24d7-4cda-b4a3-108afe9807cc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:36.395221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279bb59-24d7-4cda-b4a3-108afe9807cc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:36.395491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279bb59-24d7-4cda-b4a3-108afe9807cc None admin] PATH_INFO: `/v3/projects/93313ea135224912b47077fef26d8091` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:36.396657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279bb59-24d7-4cda-b4a3-108afe9807cc None admin] RBAC: Authorizing `identity:get_project(project_id=93313ea135224912b47077fef26d8091)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:36.398113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279bb59-24d7-4cda-b4a3-108afe9807cc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:36.399801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 580/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:36 2022] GET /identity/v3/projects/93313ea135224912b47077fef26d8091 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:36.405988 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519833dc-580d-4170-9076-d92b1ffaacfc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:36.502605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519833dc-580d-4170-9076-d92b1ffaacfc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:36.535195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17e0f9-6da8-4cc2-8a65-e1799854b165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:36.535829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17e0f9-6da8-4cc2-8a65-e1799854b165 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:36.536109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17e0f9-6da8-4cc2-8a65-e1799854b165 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:36.536382 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17e0f9-6da8-4cc2-8a65-e1799854b165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:36.537191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17e0f9-6da8-4cc2-8a65-e1799854b165 None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=93313ea135224912b47077fef26d8091, user.id=fd3fea5fa9b44faba6d2a276e51f0f65) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:36.537498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17e0f9-6da8-4cc2-8a65-e1799854b165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:36.538955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17e0f9-6da8-4cc2-8a65-e1799854b165 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:36.544387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 566/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:49:36 2022] GET /identity/v3/role_assignments?user.id=fd3fea5fa9b44faba6d2a276e51f0f65&scope.project.id=93313ea135224912b47077fef26d8091&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:38.512677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a95ae-9a04-4e43-ac70-81e429f4ee8b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:38.513193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a95ae-9a04-4e43-ac70-81e429f4ee8b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:38.513318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a95ae-9a04-4e43-ac70-81e429f4ee8b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:38.514758 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 581/1147] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:38.520033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610719b6-fbba-402b-9651-aa79bef5ab84 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:38.520450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610719b6-fbba-402b-9651-aa79bef5ab84 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:38.520601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610719b6-fbba-402b-9651-aa79bef5ab84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:38.537344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-610719b6-fbba-402b-9651-aa79bef5ab84 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:38.538557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-610719b6-fbba-402b-9651-aa79bef5ab84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:38.566904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:38.604006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 567/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:38.626744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a95ae-9a04-4e43-ac70-81e429f4ee8b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:38.724428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a95ae-9a04-4e43-ac70-81e429f4ee8b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:38.757970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258eae-31b2-4e93-a553-4b9ce9f2d462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:38.758770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258eae-31b2-4e93-a553-4b9ce9f2d462 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:38.759051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258eae-31b2-4e93-a553-4b9ce9f2d462 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:38.759322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258eae-31b2-4e93-a553-4b9ce9f2d462 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:38.770026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b258eae-31b2-4e93-a553-4b9ce9f2d462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1b258eae-31b2-4e93-a553-4b9ce9f2d462 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:38.777729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:38.778828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:38.780132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 582/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:49:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:38.784878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610719b6-fbba-402b-9651-aa79bef5ab84 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:38.880460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610719b6-fbba-402b-9651-aa79bef5ab84 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:38.912501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77927c2-6a59-4c21-9f47-856cb390c0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:38.913147 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77927c2-6a59-4c21-9f47-856cb390c0ac None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:38.913426 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77927c2-6a59-4c21-9f47-856cb390c0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:38.913697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77927c2-6a59-4c21-9f47-856cb390c0ac None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:38.914464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77927c2-6a59-4c21-9f47-856cb390c0ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:38.914770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77927c2-6a59-4c21-9f47-856cb390c0ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:38.916117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77927c2-6a59-4c21-9f47-856cb390c0ac None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:38.924712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 568/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:49:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:38.930364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258eae-31b2-4e93-a553-4b9ce9f2d462 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:38.946670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:39.027985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258eae-31b2-4e93-a553-4b9ce9f2d462 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:39.061262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8d903-a11f-4089-82f7-2391f50fe290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:39.062198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8d903-a11f-4089-82f7-2391f50fe290 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:39.062480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8d903-a11f-4089-82f7-2391f50fe290 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:39.062753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8d903-a11f-4089-82f7-2391f50fe290 None admin] PATH_INFO: `/v3/users/fd3fea5fa9b44faba6d2a276e51f0f65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:39.063954 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8d903-a11f-4089-82f7-2391f50fe290 None admin] RBAC: Authorizing `identity:get_user(user_id=fd3fea5fa9b44faba6d2a276e51f0f65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:39.065385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8d903-a11f-4089-82f7-2391f50fe290 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:39.067134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 583/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:38 2022] GET /identity/v3/users/fd3fea5fa9b44faba6d2a276e51f0f65 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:39.072810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77927c2-6a59-4c21-9f47-856cb390c0ac None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:39.100561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:39.169371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77927c2-6a59-4c21-9f47-856cb390c0ac None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:39.201931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925e4f6-49eb-486e-a1e9-044e7859d2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:39.202792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925e4f6-49eb-486e-a1e9-044e7859d2bd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:39.203068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925e4f6-49eb-486e-a1e9-044e7859d2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:39.203338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925e4f6-49eb-486e-a1e9-044e7859d2bd None admin] PATH_INFO: `/v3/projects/93313ea135224912b47077fef26d8091` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:39.204487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925e4f6-49eb-486e-a1e9-044e7859d2bd None admin] RBAC: Authorizing `identity:get_project(project_id=93313ea135224912b47077fef26d8091)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:39.205890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925e4f6-49eb-486e-a1e9-044e7859d2bd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:39.207559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 569/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:39 2022] GET /identity/v3/projects/93313ea135224912b47077fef26d8091 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:39.213457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8d903-a11f-4089-82f7-2391f50fe290 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:39.295983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:39.313747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8d903-a11f-4089-82f7-2391f50fe290 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:39.347389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc0c77-0ff4-482b-bea2-8c0e176a53c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:39.348270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc0c77-0ff4-482b-bea2-8c0e176a53c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:39.348547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc0c77-0ff4-482b-bea2-8c0e176a53c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:39.348817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc0c77-0ff4-482b-bea2-8c0e176a53c3 None admin] PATH_INFO: `/v3/projects/93313ea135224912b47077fef26d8091/users/fd3fea5fa9b44faba6d2a276e51f0f65/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:39.350726 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc0c77-0ff4-482b-bea2-8c0e176a53c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=93313ea135224912b47077fef26d8091, user_id=fd3fea5fa9b44faba6d2a276e51f0f65, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:39.352140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc0c77-0ff4-482b-bea2-8c0e176a53c3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:39.428589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 584/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:49:39 2022] PUT /identity/v3/projects/93313ea135224912b47077fef26d8091/users/fd3fea5fa9b44faba6d2a276e51f0f65/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:49:41.410597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce5178-3dc2-4521-a06c-b9360b89d39d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:41.411097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce5178-3dc2-4521-a06c-b9360b89d39d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:41.411188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce5178-3dc2-4521-a06c-b9360b89d39d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:41.412490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 570/1154] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:41.417908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2266a-8393-4983-98f3-b41596fd65b7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:41.418379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2266a-8393-4983-98f3-b41596fd65b7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:41.418487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2266a-8393-4983-98f3-b41596fd65b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:41.435650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e7d2266a-8393-4983-98f3-b41596fd65b7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:41.436813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e7d2266a-8393-4983-98f3-b41596fd65b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:41.500146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 585/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:41.522991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce5178-3dc2-4521-a06c-b9360b89d39d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:41.615574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:41.622091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce5178-3dc2-4521-a06c-b9360b89d39d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:41.654556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae8091-83b5-4217-913e-1dc77857ec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:41.655470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8091-83b5-4217-913e-1dc77857ec06 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:41.655801 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8091-83b5-4217-913e-1dc77857ec06 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:41.656374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae8091-83b5-4217-913e-1dc77857ec06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:41.666227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae8091-83b5-4217-913e-1dc77857ec06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-90ae8091-83b5-4217-913e-1dc77857ec06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:41.674652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:41.675718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:41.676804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 571/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:49:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:41.681776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2266a-8393-4983-98f3-b41596fd65b7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:41.749770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:41.777638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2266a-8393-4983-98f3-b41596fd65b7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:41.811249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7885d-ed35-485a-aac7-4b281668806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:41.812126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7885d-ed35-485a-aac7-4b281668806c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:41.812590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7885d-ed35-485a-aac7-4b281668806c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:41.813248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7885d-ed35-485a-aac7-4b281668806c None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:41.813986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7885d-ed35-485a-aac7-4b281668806c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:41.814322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7885d-ed35-485a-aac7-4b281668806c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:41.815700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7885d-ed35-485a-aac7-4b281668806c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:41.823479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 586/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:49:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:41.829191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae8091-83b5-4217-913e-1dc77857ec06 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:41.926724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae8091-83b5-4217-913e-1dc77857ec06 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:41.958923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b10c0-79bc-4eb0-bf92-27b4d26f5183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:41.959773 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b10c0-79bc-4eb0-bf92-27b4d26f5183 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:41.960051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b10c0-79bc-4eb0-bf92-27b4d26f5183 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:41.960318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b10c0-79bc-4eb0-bf92-27b4d26f5183 None admin] PATH_INFO: `/v3/users/fd3fea5fa9b44faba6d2a276e51f0f65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:41.961475 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b10c0-79bc-4eb0-bf92-27b4d26f5183 None admin] RBAC: Authorizing `identity:get_user(user_id=fd3fea5fa9b44faba6d2a276e51f0f65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:41.962926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b10c0-79bc-4eb0-bf92-27b4d26f5183 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:41.964502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 572/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:41 2022] GET /identity/v3/users/fd3fea5fa9b44faba6d2a276e51f0f65 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:41.970343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7885d-ed35-485a-aac7-4b281668806c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:42.068756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7885d-ed35-485a-aac7-4b281668806c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:42.101082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc1eb8-481f-46f2-963f-068659a0aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:42.101714 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc1eb8-481f-46f2-963f-068659a0aa0a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:42.101994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc1eb8-481f-46f2-963f-068659a0aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:42.102299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc1eb8-481f-46f2-963f-068659a0aa0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:42.104822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:42.168774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc1eb8-481f-46f2-963f-068659a0aa0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:42.170996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc1eb8-481f-46f2-963f-068659a0aa0a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:42.265744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 587/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:49:42.271552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b10c0-79bc-4eb0-bf92-27b4d26f5183 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:42.296780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:42.369669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b10c0-79bc-4eb0-bf92-27b4d26f5183 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:42.403202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26ac58-f879-4775-bbd7-8f7e18fc8264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:42.404337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ac58-f879-4775-bbd7-8f7e18fc8264 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:42.404536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ac58-f879-4775-bbd7-8f7e18fc8264 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:42.404798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ac58-f879-4775-bbd7-8f7e18fc8264 None admin] PATH_INFO: `/v3/projects/93313ea135224912b47077fef26d8091` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:42.405976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ac58-f879-4775-bbd7-8f7e18fc8264 None admin] RBAC: Authorizing `identity:get_project(project_id=93313ea135224912b47077fef26d8091)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:42.407442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ac58-f879-4775-bbd7-8f7e18fc8264 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:42.409111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 573/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:42 2022] GET /identity/v3/projects/93313ea135224912b47077fef26d8091 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:42.415120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc1eb8-481f-46f2-963f-068659a0aa0a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:42.447533 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:42.510168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc1eb8-481f-46f2-963f-068659a0aa0a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:42.542923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e187dc8-8f40-42f6-ba15-bfd6942b6271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:42.543505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e187dc8-8f40-42f6-ba15-bfd6942b6271 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:42.543783 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e187dc8-8f40-42f6-ba15-bfd6942b6271 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:42.544054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e187dc8-8f40-42f6-ba15-bfd6942b6271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:42.545395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e187dc8-8f40-42f6-ba15-bfd6942b6271 None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=93313ea135224912b47077fef26d8091, user.id=fd3fea5fa9b44faba6d2a276e51f0f65) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:42.545700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e187dc8-8f40-42f6-ba15-bfd6942b6271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:42.547145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e187dc8-8f40-42f6-ba15-bfd6942b6271 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:42.552754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 588/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:49:42 2022] GET /identity/v3/role_assignments?user.id=fd3fea5fa9b44faba6d2a276e51f0f65&scope.project.id=93313ea135224912b47077fef26d8091&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:44.574880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a34b21-2381-47e8-8bf8-c81c59b7f1cf None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:44.575396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a34b21-2381-47e8-8bf8-c81c59b7f1cf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:44.575396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a34b21-2381-47e8-8bf8-c81c59b7f1cf None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:44.576902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 574/1162] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:44.582456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924362d-36a1-43d5-a85a-1e8eef9e7487 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:44.582868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924362d-36a1-43d5-a85a-1e8eef9e7487 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:44.583014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924362d-36a1-43d5-a85a-1e8eef9e7487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:44.600759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6924362d-36a1-43d5-a85a-1e8eef9e7487 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:44.601871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6924362d-36a1-43d5-a85a-1e8eef9e7487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:44.666811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 589/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:44.689674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a34b21-2381-47e8-8bf8-c81c59b7f1cf None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:44.779196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:44.788185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a34b21-2381-47e8-8bf8-c81c59b7f1cf None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:44.821985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72560659-ad39-45f7-ae46-da068a7c111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:44.822650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72560659-ad39-45f7-ae46-da068a7c111e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:44.822933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72560659-ad39-45f7-ae46-da068a7c111e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:44.823211 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72560659-ad39-45f7-ae46-da068a7c111e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:44.887205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72560659-ad39-45f7-ae46-da068a7c111e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:44.888888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72560659-ad39-45f7-ae46-da068a7c111e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:44.976011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:44.985015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 575/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:49:44.990976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924362d-36a1-43d5-a85a-1e8eef9e7487 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:45.018702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:45.088143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924362d-36a1-43d5-a85a-1e8eef9e7487 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:45.121348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6eb64-fd0e-49fe-bf5c-4260e355b3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:45.122164 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6eb64-fd0e-49fe-bf5c-4260e355b3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:45.122482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6eb64-fd0e-49fe-bf5c-4260e355b3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:45.122760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6eb64-fd0e-49fe-bf5c-4260e355b3a0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:45.129275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6eb64-fd0e-49fe-bf5c-4260e355b3a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:45.130744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6eb64-fd0e-49fe-bf5c-4260e355b3a0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-48b6eb64-fd0e-49fe-bf5c-4260e355b3a0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:45.145451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:45.146752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:49:45.147799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 590/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 13 08:49:44 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:45.152513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72560659-ad39-45f7-ae46-da068a7c111e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:45.248565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72560659-ad39-45f7-ae46-da068a7c111e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:45.281181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df64d0-fcfb-4ed9-a011-c976b1f53503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:45.281826 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df64d0-fcfb-4ed9-a011-c976b1f53503 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:45.282136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df64d0-fcfb-4ed9-a011-c976b1f53503 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:45.282414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df64d0-fcfb-4ed9-a011-c976b1f53503 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:45.283118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df64d0-fcfb-4ed9-a011-c976b1f53503 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:45.283420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df64d0-fcfb-4ed9-a011-c976b1f53503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:45.284764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df64d0-fcfb-4ed9-a011-c976b1f53503 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:45.291622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 576/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 13 08:49:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:47.298521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3b509-b5f4-4fd0-bef0-16caafed063c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:47.299048 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3b509-b5f4-4fd0-bef0-16caafed063c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:47.299168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3b509-b5f4-4fd0-bef0-16caafed063c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:47.300443 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 591/1167] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:47.305612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa826e-1cb9-46ec-80eb-906a97b986c8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:47.306024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa826e-1cb9-46ec-80eb-906a97b986c8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:47.306198 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa826e-1cb9-46ec-80eb-906a97b986c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:47.323077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7cfa826e-1cb9-46ec-80eb-906a97b986c8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:47.324245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7cfa826e-1cb9-46ec-80eb-906a97b986c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:47.375818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:47.391613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 577/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 08:49:47.415184 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3b509-b5f4-4fd0-bef0-16caafed063c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:47.502162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:47.514340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3b509-b5f4-4fd0-bef0-16caafed063c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:47.548301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767daee7-2f19-450b-a1e1-6dd900b36199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:47.548991 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767daee7-2f19-450b-a1e1-6dd900b36199 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:47.549220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767daee7-2f19-450b-a1e1-6dd900b36199 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:47.549496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767daee7-2f19-450b-a1e1-6dd900b36199 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:47.550248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767daee7-2f19-450b-a1e1-6dd900b36199 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:47.551606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767daee7-2f19-450b-a1e1-6dd900b36199 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:47.602893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 592/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Wed Jul 13 08:49:47 2022] POST /identity/v3/domains => generated 267 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:49:49.630252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d093036-ffa4-4b6e-9737-3ec97832f063 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:49.630768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d093036-ffa4-4b6e-9737-3ec97832f063 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:49.630859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d093036-ffa4-4b6e-9737-3ec97832f063 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:49.632159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 578/1170] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:49.637162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731fe9-a877-434f-a7c0-cda81141bab3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:49.637590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731fe9-a877-434f-a7c0-cda81141bab3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:49.637722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731fe9-a877-434f-a7c0-cda81141bab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:49.654729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-74731fe9-a877-434f-a7c0-cda81141bab3 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:49.655927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-74731fe9-a877-434f-a7c0-cda81141bab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:49.720801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 593/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:49.744292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d093036-ffa4-4b6e-9737-3ec97832f063 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:49.842912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d093036-ffa4-4b6e-9737-3ec97832f063 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:49.874970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe07ed-3b33-40db-bc00-eb29ced85059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:49.875805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe07ed-3b33-40db-bc00-eb29ced85059 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:49.876087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe07ed-3b33-40db-bc00-eb29ced85059 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:49.876363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe07ed-3b33-40db-bc00-eb29ced85059 None admin] PATH_INFO: `/v3/domains/a9d562013ee6499a9691e7ebb3ad301a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:49.886786 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe07ed-3b33-40db-bc00-eb29ced85059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a9d562013ee6499a9691e7ebb3ad301a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:49.888745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe07ed-3b33-40db-bc00-eb29ced85059 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:49.890397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 579/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Wed Jul 13 08:49:49 2022] GET /identity/v3/domains/a9d562013ee6499a9691e7ebb3ad301a => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:49.896156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74731fe9-a877-434f-a7c0-cda81141bab3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:49.972295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:49.992663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74731fe9-a877-434f-a7c0-cda81141bab3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:50.025668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8bb27-36c8-43df-8a94-2135090ad13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:50.026357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8bb27-36c8-43df-8a94-2135090ad13a None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:50.026644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8bb27-36c8-43df-8a94-2135090ad13a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:50.026922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8bb27-36c8-43df-8a94-2135090ad13a None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:50.027634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8bb27-36c8-43df-8a94-2135090ad13a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:50.029694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8bb27-36c8-43df-8a94-2135090ad13a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:50.134343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 594/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:49:49 2022] POST /identity/v3/projects => generated 361 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:49:52.173041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046185a2-7cfd-41a6-8d15-712eb36df0b6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:52.173575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046185a2-7cfd-41a6-8d15-712eb36df0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:52.173575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046185a2-7cfd-41a6-8d15-712eb36df0b6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:52.174972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 580/1174] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:52.180155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca19c-2951-4706-a482-64b830b99476 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:52.180552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca19c-2951-4706-a482-64b830b99476 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:52.180715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca19c-2951-4706-a482-64b830b99476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:52.197699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4f9ca19c-2951-4706-a482-64b830b99476 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:52.198904 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4f9ca19c-2951-4706-a482-64b830b99476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:52.264036 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 595/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:52.288054 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046185a2-7cfd-41a6-8d15-712eb36df0b6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:52.335672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:52.388756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046185a2-7cfd-41a6-8d15-712eb36df0b6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:52.421693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b357d-1d74-47a7-9b61-7cae2a11e252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:52.422575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b357d-1d74-47a7-9b61-7cae2a11e252 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:52.422858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b357d-1d74-47a7-9b61-7cae2a11e252 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:52.423185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b357d-1d74-47a7-9b61-7cae2a11e252 None admin] PATH_INFO: `/v3/domains/a9d562013ee6499a9691e7ebb3ad301a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:52.424307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b357d-1d74-47a7-9b61-7cae2a11e252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a9d562013ee6499a9691e7ebb3ad301a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:52.425726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b357d-1d74-47a7-9b61-7cae2a11e252 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:52.427506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 581/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Wed Jul 13 08:49:52 2022] GET /identity/v3/domains/a9d562013ee6499a9691e7ebb3ad301a => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:52.433194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ca19c-2951-4706-a482-64b830b99476 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:52.490624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:52.531407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ca19c-2951-4706-a482-64b830b99476 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:52.565123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79b22c-77ba-4f98-832e-e8b40a6886dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:52.565772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79b22c-77ba-4f98-832e-e8b40a6886dd None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:52.566070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79b22c-77ba-4f98-832e-e8b40a6886dd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:52.566344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79b22c-77ba-4f98-832e-e8b40a6886dd None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:52.567045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79b22c-77ba-4f98-832e-e8b40a6886dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:52.568458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79b22c-77ba-4f98-832e-e8b40a6886dd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:52.639721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 596/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:49:52 2022] POST /identity/v3/users => generated 311 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:49:54.716492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0ebc9-f11d-4953-a59e-97b123ab601d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:54.717069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0ebc9-f11d-4953-a59e-97b123ab601d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:54.717342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0ebc9-f11d-4953-a59e-97b123ab601d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:54.718726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 582/1178] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:54.724028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deda116-8cf6-4888-8284-567ead7fb14b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:54.724451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deda116-8cf6-4888-8284-567ead7fb14b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:54.724579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deda116-8cf6-4888-8284-567ead7fb14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:54.741762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9deda116-8cf6-4888-8284-567ead7fb14b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:54.742950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9deda116-8cf6-4888-8284-567ead7fb14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:54.811000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 597/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:54.834420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0ebc9-f11d-4953-a59e-97b123ab601d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:54.932203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0ebc9-f11d-4953-a59e-97b123ab601d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:54.946094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:54.965307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf93aa8-a19d-4ec7-8efe-ce81b2340b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:54.966118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf93aa8-a19d-4ec7-8efe-ce81b2340b37 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:54.966397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf93aa8-a19d-4ec7-8efe-ce81b2340b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:54.966711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf93aa8-a19d-4ec7-8efe-ce81b2340b37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:54.976279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf93aa8-a19d-4ec7-8efe-ce81b2340b37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-1cf93aa8-a19d-4ec7-8efe-ce81b2340b37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:54.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:54.985960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:54.986945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 583/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:49:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:54.992026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deda116-8cf6-4888-8284-567ead7fb14b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:55.036930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:55.088372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deda116-8cf6-4888-8284-567ead7fb14b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:55.122698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdaebf-2fcd-4318-aafc-8822b388d89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:55.123351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdaebf-2fcd-4318-aafc-8822b388d89b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:55.123633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdaebf-2fcd-4318-aafc-8822b388d89b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:55.123907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdaebf-2fcd-4318-aafc-8822b388d89b None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:55.124645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdaebf-2fcd-4318-aafc-8822b388d89b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:55.124948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdaebf-2fcd-4318-aafc-8822b388d89b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:55.126381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdaebf-2fcd-4318-aafc-8822b388d89b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:55.134320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 598/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:49:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:55.140004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf93aa8-a19d-4ec7-8efe-ce81b2340b37 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:55.237306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf93aa8-a19d-4ec7-8efe-ce81b2340b37 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:55.270767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0f9c3-f04a-4c47-a64f-cc4ae77e11fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:55.271612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0f9c3-f04a-4c47-a64f-cc4ae77e11fc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:55.271888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0f9c3-f04a-4c47-a64f-cc4ae77e11fc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:55.272157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0f9c3-f04a-4c47-a64f-cc4ae77e11fc None admin] PATH_INFO: `/v3/users/ec9ddfef3f2d4102899759b9c56fdc13` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:55.285257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0f9c3-f04a-4c47-a64f-cc4ae77e11fc None admin] RBAC: Authorizing `identity:get_user(user_id=ec9ddfef3f2d4102899759b9c56fdc13)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:55.286705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0f9c3-f04a-4c47-a64f-cc4ae77e11fc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:55.288316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 584/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:55 2022] GET /identity/v3/users/ec9ddfef3f2d4102899759b9c56fdc13 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:55.295340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdaebf-2fcd-4318-aafc-8822b388d89b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:55.394282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdaebf-2fcd-4318-aafc-8822b388d89b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:55.402676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:55.428133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc3bbe-a1b1-4f29-b724-a01091ba8552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:55.428762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc3bbe-a1b1-4f29-b724-a01091ba8552 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:55.429077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc3bbe-a1b1-4f29-b724-a01091ba8552 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:55.429358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc3bbe-a1b1-4f29-b724-a01091ba8552 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:55.495163 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc3bbe-a1b1-4f29-b724-a01091ba8552 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:55.496570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc3bbe-a1b1-4f29-b724-a01091ba8552 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:55.592201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 599/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:49:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:49:55.598169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0f9c3-f04a-4c47-a64f-cc4ae77e11fc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:55.628373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:55.694878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0f9c3-f04a-4c47-a64f-cc4ae77e11fc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:55.728547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fecb3-3aa7-413c-91a2-6432ecf699e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:55.729390 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fecb3-3aa7-413c-91a2-6432ecf699e8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:55.729666 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fecb3-3aa7-413c-91a2-6432ecf699e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:55.729937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fecb3-3aa7-413c-91a2-6432ecf699e8 None admin] PATH_INFO: `/v3/projects/a49282518e3a49da9200ab48ef422807` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:55.731129 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fecb3-3aa7-413c-91a2-6432ecf699e8 None admin] RBAC: Authorizing `identity:get_project(project_id=a49282518e3a49da9200ab48ef422807)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:55.732726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fecb3-3aa7-413c-91a2-6432ecf699e8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:55.734404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 585/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:55 2022] GET /identity/v3/projects/a49282518e3a49da9200ab48ef422807 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:55.740299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc3bbe-a1b1-4f29-b724-a01091ba8552 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:55.750462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:55.837727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc3bbe-a1b1-4f29-b724-a01091ba8552 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:55.870570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544914d-8760-42fb-bb02-9ddc0ec2d7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:55.871194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544914d-8760-42fb-bb02-9ddc0ec2d7ba None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:55.871471 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544914d-8760-42fb-bb02-9ddc0ec2d7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:55.871742 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544914d-8760-42fb-bb02-9ddc0ec2d7ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:55.872880 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544914d-8760-42fb-bb02-9ddc0ec2d7ba None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=a49282518e3a49da9200ab48ef422807, user.id=ec9ddfef3f2d4102899759b9c56fdc13) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:55.873188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544914d-8760-42fb-bb02-9ddc0ec2d7ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:55.874628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544914d-8760-42fb-bb02-9ddc0ec2d7ba None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:55.879789 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 600/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:49:55 2022] GET /identity/v3/role_assignments?user.id=ec9ddfef3f2d4102899759b9c56fdc13&scope.project.id=a49282518e3a49da9200ab48ef422807&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:57.875263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5fa60-8c1b-493c-a5a4-e9bac273c31d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:57.875830 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5fa60-8c1b-493c-a5a4-e9bac273c31d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:57.875830 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5fa60-8c1b-493c-a5a4-e9bac273c31d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:57.877153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 586/1186] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:49:57.882520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b1bbb-bc82-4004-a4c2-ca63bab2583c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:57.882905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b1bbb-bc82-4004-a4c2-ca63bab2583c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:57.883066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b1bbb-bc82-4004-a4c2-ca63bab2583c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:57.900045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b68b1bbb-bc82-4004-a4c2-ca63bab2583c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:49:57.901207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b68b1bbb-bc82-4004-a4c2-ca63bab2583c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:49:57.965049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 601/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:49:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:49:57.987597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5fa60-8c1b-493c-a5a4-e9bac273c31d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:58.082978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:58.086691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5fa60-8c1b-493c-a5a4-e9bac273c31d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:58.119188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f11d-31a9-4ecf-9dc5-e468708035a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:58.119958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f11d-31a9-4ecf-9dc5-e468708035a9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:58.120241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f11d-31a9-4ecf-9dc5-e468708035a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:58.120516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f11d-31a9-4ecf-9dc5-e468708035a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:58.131610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0f11d-31a9-4ecf-9dc5-e468708035a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f0e0f11d-31a9-4ecf-9dc5-e468708035a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:49:58.139223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:49:58.140307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:49:58.141188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 587/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:49:58.146525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b1bbb-bc82-4004-a4c2-ca63bab2583c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:58.148681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:58.242869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b1bbb-bc82-4004-a4c2-ca63bab2583c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:58.275856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf54ce-a0ae-49b1-aa1f-d07b721ef624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:58.276509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf54ce-a0ae-49b1-aa1f-d07b721ef624 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:58.276790 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf54ce-a0ae-49b1-aa1f-d07b721ef624 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:58.277061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf54ce-a0ae-49b1-aa1f-d07b721ef624 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:58.277795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf54ce-a0ae-49b1-aa1f-d07b721ef624 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:49:58.278230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf54ce-a0ae-49b1-aa1f-d07b721ef624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:58.279598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf54ce-a0ae-49b1-aa1f-d07b721ef624 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:58.287419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 602/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:49:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:58.293393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f11d-31a9-4ecf-9dc5-e468708035a9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:58.393177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f11d-31a9-4ecf-9dc5-e468708035a9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:58.424747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c97871-02cc-4431-84b3-30f31820d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:58.425595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c97871-02cc-4431-84b3-30f31820d4af None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:58.425872 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c97871-02cc-4431-84b3-30f31820d4af None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:58.426179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c97871-02cc-4431-84b3-30f31820d4af None admin] PATH_INFO: `/v3/users/ec9ddfef3f2d4102899759b9c56fdc13` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:58.427329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c97871-02cc-4431-84b3-30f31820d4af None admin] RBAC: Authorizing `identity:get_user(user_id=ec9ddfef3f2d4102899759b9c56fdc13)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:58.428737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c97871-02cc-4431-84b3-30f31820d4af None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:58.430346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 588/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:49:58 2022] GET /identity/v3/users/ec9ddfef3f2d4102899759b9c56fdc13 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:58.436611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf54ce-a0ae-49b1-aa1f-d07b721ef624 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:58.496332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:58.533125 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf54ce-a0ae-49b1-aa1f-d07b721ef624 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:58.565950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677a3b0-995d-4b55-b6e8-7868a1602f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:58.566828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677a3b0-995d-4b55-b6e8-7868a1602f74 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:58.567114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677a3b0-995d-4b55-b6e8-7868a1602f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:58.567390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677a3b0-995d-4b55-b6e8-7868a1602f74 None admin] PATH_INFO: `/v3/projects/a49282518e3a49da9200ab48ef422807` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:58.568558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677a3b0-995d-4b55-b6e8-7868a1602f74 None admin] RBAC: Authorizing `identity:get_project(project_id=a49282518e3a49da9200ab48ef422807)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:58.569990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677a3b0-995d-4b55-b6e8-7868a1602f74 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:58.571707 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 603/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:49:58 2022] GET /identity/v3/projects/a49282518e3a49da9200ab48ef422807 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:49:58.578293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c97871-02cc-4431-84b3-30f31820d4af None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:49:58.605706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:49:58.675291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c97871-02cc-4431-84b3-30f31820d4af None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:49:58.708369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d16968-e603-41da-8e7f-43e42addb6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:49:58.709230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d16968-e603-41da-8e7f-43e42addb6db None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:49:58.709505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d16968-e603-41da-8e7f-43e42addb6db None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:49:58.709773 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d16968-e603-41da-8e7f-43e42addb6db None admin] PATH_INFO: `/v3/projects/a49282518e3a49da9200ab48ef422807/users/ec9ddfef3f2d4102899759b9c56fdc13/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:49:58.711712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d16968-e603-41da-8e7f-43e42addb6db None admin] RBAC: Authorizing `identity:create_grant(project_id=a49282518e3a49da9200ab48ef422807, user_id=ec9ddfef3f2d4102899759b9c56fdc13, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:49:58.713115 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d16968-e603-41da-8e7f-43e42addb6db None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:49:58.790097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 589/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:49:58 2022] PUT /identity/v3/projects/a49282518e3a49da9200ab48ef422807/users/ec9ddfef3f2d4102899759b9c56fdc13/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:50:00.779565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87666c-976b-40c5-b2de-60d5ef6c6661 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:50:00.780120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87666c-976b-40c5-b2de-60d5ef6c6661 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:50:00.780120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87666c-976b-40c5-b2de-60d5ef6c6661 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:50:00.781480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 604/1193] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:50:00.786535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09726854-ba1b-4fcd-80ee-7bf061c9a0c7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:50:00.786950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09726854-ba1b-4fcd-80ee-7bf061c9a0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:50:00.787084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09726854-ba1b-4fcd-80ee-7bf061c9a0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:50:00.803597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-09726854-ba1b-4fcd-80ee-7bf061c9a0c7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:50:00.804773 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-09726854-ba1b-4fcd-80ee-7bf061c9a0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:50:00.868524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 590/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:50:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:50:00.892115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87666c-976b-40c5-b2de-60d5ef6c6661 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:50:00.990836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87666c-976b-40c5-b2de-60d5ef6c6661 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:50:01.018665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:50:01.023256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a961c7-7e99-4b03-9acb-b56ee0a2d97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:50:01.024033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a961c7-7e99-4b03-9acb-b56ee0a2d97d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:50:01.024314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a961c7-7e99-4b03-9acb-b56ee0a2d97d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:50:01.024586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a961c7-7e99-4b03-9acb-b56ee0a2d97d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:50:01.041090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a961c7-7e99-4b03-9acb-b56ee0a2d97d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-80a961c7-7e99-4b03-9acb-b56ee0a2d97d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:50:01.042798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:50:01.049457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:50:01.050563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 605/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:50:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:50:01.051443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09726854-ba1b-4fcd-80ee-7bf061c9a0c7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:50:01.069337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:50:01.146950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09726854-ba1b-4fcd-80ee-7bf061c9a0c7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:50:01.179926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da6049-2c8d-4fb0-8f56-edbc2108e85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:50:01.180583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6049-2c8d-4fb0-8f56-edbc2108e85b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:50:01.180865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6049-2c8d-4fb0-8f56-edbc2108e85b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:50:01.181136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6049-2c8d-4fb0-8f56-edbc2108e85b None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:50:01.181868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da6049-2c8d-4fb0-8f56-edbc2108e85b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:50:01.182198 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da6049-2c8d-4fb0-8f56-edbc2108e85b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:50:01.183646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da6049-2c8d-4fb0-8f56-edbc2108e85b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:50:01.192656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 591/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:50:01.198459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a961c7-7e99-4b03-9acb-b56ee0a2d97d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:50:01.296537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a961c7-7e99-4b03-9acb-b56ee0a2d97d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:50:01.329706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40175e-5382-44d3-89c3-7daa348fe882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:50:01.330585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40175e-5382-44d3-89c3-7daa348fe882 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:50:01.330864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40175e-5382-44d3-89c3-7daa348fe882 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:50:01.331135 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40175e-5382-44d3-89c3-7daa348fe882 None admin] PATH_INFO: `/v3/users/ec9ddfef3f2d4102899759b9c56fdc13` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:50:01.332291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40175e-5382-44d3-89c3-7daa348fe882 None admin] RBAC: Authorizing `identity:get_user(user_id=ec9ddfef3f2d4102899759b9c56fdc13)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:50:01.333705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40175e-5382-44d3-89c3-7daa348fe882 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:50:01.335945 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 606/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 13 08:50:01 2022] GET /identity/v3/users/ec9ddfef3f2d4102899759b9c56fdc13 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:50:01.342201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da6049-2c8d-4fb0-8f56-edbc2108e85b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:50:01.422490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:50:01.440890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da6049-2c8d-4fb0-8f56-edbc2108e85b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:50:01.474209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2559acf-14b2-4c42-9299-472ace8d32f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:50:01.474841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2559acf-14b2-4c42-9299-472ace8d32f5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:50:01.475121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2559acf-14b2-4c42-9299-472ace8d32f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:50:01.475394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2559acf-14b2-4c42-9299-472ace8d32f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:50:01.540710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2559acf-14b2-4c42-9299-472ace8d32f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:50:01.542439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2559acf-14b2-4c42-9299-472ace8d32f5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:50:01.618047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:50:01.636297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 592/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:50:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:50:01.642270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40175e-5382-44d3-89c3-7daa348fe882 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:50:01.700243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:50:01.739362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40175e-5382-44d3-89c3-7daa348fe882 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:50:01.772066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac96325-d6c0-44d6-a1d2-15302ee0c627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:50:01.772906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac96325-d6c0-44d6-a1d2-15302ee0c627 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:50:01.773182 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac96325-d6c0-44d6-a1d2-15302ee0c627 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:50:01.773453 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac96325-d6c0-44d6-a1d2-15302ee0c627 None admin] PATH_INFO: `/v3/projects/a49282518e3a49da9200ab48ef422807` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:50:01.774630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac96325-d6c0-44d6-a1d2-15302ee0c627 None admin] RBAC: Authorizing `identity:get_project(project_id=a49282518e3a49da9200ab48ef422807)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:50:01.776049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac96325-d6c0-44d6-a1d2-15302ee0c627 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:50:01.777709 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 607/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 13 08:50:01 2022] GET /identity/v3/projects/a49282518e3a49da9200ab48ef422807 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:50:01.784470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2559acf-14b2-4c42-9299-472ace8d32f5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:50:01.879333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2559acf-14b2-4c42-9299-472ace8d32f5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:50:01.912321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3bb37-02c8-4566-a245-5fb1f0874f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:50:01.912956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3bb37-02c8-4566-a245-5fb1f0874f8f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:50:01.913236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3bb37-02c8-4566-a245-5fb1f0874f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:50:01.913508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3bb37-02c8-4566-a245-5fb1f0874f8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:50:01.914351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3bb37-02c8-4566-a245-5fb1f0874f8f None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=a49282518e3a49da9200ab48ef422807, user.id=ec9ddfef3f2d4102899759b9c56fdc13) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:50:01.914657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3bb37-02c8-4566-a245-5fb1f0874f8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:50:01.916062 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3bb37-02c8-4566-a245-5fb1f0874f8f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:50:01.922015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 593/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:50:01 2022] GET /identity/v3/role_assignments?user.id=ec9ddfef3f2d4102899759b9c56fdc13&scope.project.id=a49282518e3a49da9200ab48ef422807&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:50:11.768120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=36 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:50:11.919453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=78 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:14.363150 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e468-007a-4915-8ced-a8387973a795 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:14.369122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e468-007a-4915-8ced-a8387973a795 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:14.369122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e468-007a-4915-8ced-a8387973a795 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:14.369122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 608/1201] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:14.370251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9af8de-14b7-4a23-9ee6-0df677901258 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:14.370657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9af8de-14b7-4a23-9ee6-0df677901258 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:14.370807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9af8de-14b7-4a23-9ee6-0df677901258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:14.388281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-db9af8de-14b7-4a23-9ee6-0df677901258 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:14.392074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-db9af8de-14b7-4a23-9ee6-0df677901258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:14.456634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 594/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:14.481212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4e468-007a-4915-8ced-a8387973a795 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:14.580720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4e468-007a-4915-8ced-a8387973a795 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:14.615000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7c13-4220-40aa-b3f7-983ed926c3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:14.615785 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7c13-4220-40aa-b3f7-983ed926c3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:14.616070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7c13-4220-40aa-b3f7-983ed926c3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:14.616348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7c13-4220-40aa-b3f7-983ed926c3a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:14.626352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b7c13-4220-40aa-b3f7-983ed926c3a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:14.627788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b7c13-4220-40aa-b3f7-983ed926c3a7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-bb8b7c13-4220-40aa-b3f7-983ed926c3a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:14.645080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:14.646225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:14.647226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:14.647226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:14.647226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:14.647226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:14.647226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 609/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:14.652295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9af8de-14b7-4a23-9ee6-0df677901258 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:14.752725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9af8de-14b7-4a23-9ee6-0df677901258 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:14.782590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:14.788336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd3f28-7136-44fe-86e9-1230b8303ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:14.789336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd3f28-7136-44fe-86e9-1230b8303ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:14.789634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd3f28-7136-44fe-86e9-1230b8303ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:14.789910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd3f28-7136-44fe-86e9-1230b8303ba3 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:14.790658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd3f28-7136-44fe-86e9-1230b8303ba3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:14.790965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd3f28-7136-44fe-86e9-1230b8303ba3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:14.792327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd3f28-7136-44fe-86e9-1230b8303ba3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:14.803482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 595/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:14.809299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7c13-4220-40aa-b3f7-983ed926c3a7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:14.862555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:14.907802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7c13-4220-40aa-b3f7-983ed926c3a7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:14.940756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf19f64-e7e8-4ec5-8fe9-a76e089094dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:14.941418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf19f64-e7e8-4ec5-8fe9-a76e089094dc None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:14.941701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf19f64-e7e8-4ec5-8fe9-a76e089094dc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:14.941977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf19f64-e7e8-4ec5-8fe9-a76e089094dc None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:14.942701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf19f64-e7e8-4ec5-8fe9-a76e089094dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:14.944126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf19f64-e7e8-4ec5-8fe9-a76e089094dc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:15.038129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 610/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 13 08:51:14 2022] POST /identity/v3/users => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:51:17.075571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa8429-fc62-4c50-9ab9-2c0140c926c7 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:17.076148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa8429-fc62-4c50-9ab9-2c0140c926c7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:17.076251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa8429-fc62-4c50-9ab9-2c0140c926c7 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:17.077538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 596/1206] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:17.082566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fedd98-0408-415e-ab15-5fd948123a62 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:17.083054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fedd98-0408-415e-ab15-5fd948123a62 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:17.083186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fedd98-0408-415e-ab15-5fd948123a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:17.099894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-64fedd98-0408-415e-ab15-5fd948123a62 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:17.105814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-64fedd98-0408-415e-ab15-5fd948123a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:17.169513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 611/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:17.192405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa8429-fc62-4c50-9ab9-2c0140c926c7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:17.297422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa8429-fc62-4c50-9ab9-2c0140c926c7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:17.332132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85bf83-2705-4c44-9616-daafa90dfff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:17.332846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85bf83-2705-4c44-9616-daafa90dfff5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:17.333132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85bf83-2705-4c44-9616-daafa90dfff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:17.333402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85bf83-2705-4c44-9616-daafa90dfff5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:17.347604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85bf83-2705-4c44-9616-daafa90dfff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-4d85bf83-2705-4c44-9616-daafa90dfff5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:51:17.355329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:17.356491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:17.357842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 597/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:51:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:17.362596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fedd98-0408-415e-ab15-5fd948123a62 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:17.404326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:17.461083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fedd98-0408-415e-ab15-5fd948123a62 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:17.493672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297882d5-4588-4869-92fd-3779272a9f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:17.494360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297882d5-4588-4869-92fd-3779272a9f01 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:17.494699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297882d5-4588-4869-92fd-3779272a9f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:17.494978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297882d5-4588-4869-92fd-3779272a9f01 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:17.495721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297882d5-4588-4869-92fd-3779272a9f01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:17.496028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297882d5-4588-4869-92fd-3779272a9f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:17.497903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297882d5-4588-4869-92fd-3779272a9f01 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:17.510208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 612/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:51:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:17.511655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85bf83-2705-4c44-9616-daafa90dfff5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:17.562621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:17.611965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85bf83-2705-4c44-9616-daafa90dfff5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:17.645535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131aeba0-e09d-4bf4-babe-52aa80d80094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:17.646363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131aeba0-e09d-4bf4-babe-52aa80d80094 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:17.646644 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131aeba0-e09d-4bf4-babe-52aa80d80094 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:17.646917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131aeba0-e09d-4bf4-babe-52aa80d80094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:17.657476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131aeba0-e09d-4bf4-babe-52aa80d80094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:17.658937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131aeba0-e09d-4bf4-babe-52aa80d80094 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-131aeba0-e09d-4bf4-babe-52aa80d80094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:17.676214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:17.677290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:17.678473 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:17.678473 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:17.678473 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:17.678473 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:17.678473 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 598/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:17.683294 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297882d5-4588-4869-92fd-3779272a9f01 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:17.782574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:17.785405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297882d5-4588-4869-92fd-3779272a9f01 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:17.819147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddf5a9-1498-41b7-8f77-4ce73e662a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:17.819799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddf5a9-1498-41b7-8f77-4ce73e662a80 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:17.820076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddf5a9-1498-41b7-8f77-4ce73e662a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:17.820345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddf5a9-1498-41b7-8f77-4ce73e662a80 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:17.821136 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddf5a9-1498-41b7-8f77-4ce73e662a80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:17.821431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddf5a9-1498-41b7-8f77-4ce73e662a80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:17.822853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddf5a9-1498-41b7-8f77-4ce73e662a80 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:17.834463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 613/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:17.840069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131aeba0-e09d-4bf4-babe-52aa80d80094 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:17.940754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131aeba0-e09d-4bf4-babe-52aa80d80094 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:17.942212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:17.974005 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f63c79-3d6b-4f90-b5e3-cfb9ea5ac262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:17.974944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63c79-3d6b-4f90-b5e3-cfb9ea5ac262 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:17.975226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63c79-3d6b-4f90-b5e3-cfb9ea5ac262 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:17.975494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63c79-3d6b-4f90-b5e3-cfb9ea5ac262 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:17.983469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f63c79-3d6b-4f90-b5e3-cfb9ea5ac262 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:17.984887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f63c79-3d6b-4f90-b5e3-cfb9ea5ac262 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-56f63c79-3d6b-4f90-b5e3-cfb9ea5ac262 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:51:18.001233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:18.002449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:18.003444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 599/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:18.008296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddf5a9-1498-41b7-8f77-4ce73e662a80 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:18.105711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddf5a9-1498-41b7-8f77-4ce73e662a80 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:18.138910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618f77b-1a90-496e-8cb1-b9cd0db19680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:18.139562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f77b-1a90-496e-8cb1-b9cd0db19680 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:18.139841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f77b-1a90-496e-8cb1-b9cd0db19680 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:18.140113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618f77b-1a90-496e-8cb1-b9cd0db19680 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:18.140927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618f77b-1a90-496e-8cb1-b9cd0db19680 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:18.141232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618f77b-1a90-496e-8cb1-b9cd0db19680 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:18.142699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618f77b-1a90-496e-8cb1-b9cd0db19680 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:18.159657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 614/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 13 08:51:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:18.165601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f63c79-3d6b-4f90-b5e3-cfb9ea5ac262 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:18.263875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f63c79-3d6b-4f90-b5e3-cfb9ea5ac262 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:18.298812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9ab76-6d06-4816-80dd-b3f007016f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:18.299452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9ab76-6d06-4816-80dd-b3f007016f9c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:18.299736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9ab76-6d06-4816-80dd-b3f007016f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:18.300013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9ab76-6d06-4816-80dd-b3f007016f9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:18.329755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:18.368126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9ab76-6d06-4816-80dd-b3f007016f9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:18.369733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9ab76-6d06-4816-80dd-b3f007016f9c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:18.467300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 600/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:51:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:51:18.473089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618f77b-1a90-496e-8cb1-b9cd0db19680 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:18.492196 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:18.577834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618f77b-1a90-496e-8cb1-b9cd0db19680 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:18.611328 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb8b78-2589-447c-8a5f-6e496a2364a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:18.612104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb8b78-2589-447c-8a5f-6e496a2364a3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:18.612381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb8b78-2589-447c-8a5f-6e496a2364a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:18.612651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb8b78-2589-447c-8a5f-6e496a2364a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:18.622839 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb8b78-2589-447c-8a5f-6e496a2364a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:18.624261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb8b78-2589-447c-8a5f-6e496a2364a3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7bcb8b78-2589-447c-8a5f-6e496a2364a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:18.642362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:18.643680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:18.645034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:18.645034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:18.645034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:18.645034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:18.645034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 615/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:18.649271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9ab76-6d06-4816-80dd-b3f007016f9c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:18.710177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:18.748108 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9ab76-6d06-4816-80dd-b3f007016f9c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:18.781240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a669c-ff75-431f-83ca-99ffba62ce8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:18.782126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a669c-ff75-431f-83ca-99ffba62ce8c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:18.782817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a669c-ff75-431f-83ca-99ffba62ce8c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:18.783087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a669c-ff75-431f-83ca-99ffba62ce8c None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:18.783789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a669c-ff75-431f-83ca-99ffba62ce8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:18.784091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a669c-ff75-431f-83ca-99ffba62ce8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:18.785432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a669c-ff75-431f-83ca-99ffba62ce8c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:18.797310 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 601/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:18.803198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb8b78-2589-447c-8a5f-6e496a2364a3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:18.871666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:18.902184 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb8b78-2589-447c-8a5f-6e496a2364a3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:18.935750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301af50-9a93-47ef-9862-2232b72636b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:18.936521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301af50-9a93-47ef-9862-2232b72636b5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:18.936803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301af50-9a93-47ef-9862-2232b72636b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:18.937076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301af50-9a93-47ef-9862-2232b72636b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:18.943621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301af50-9a93-47ef-9862-2232b72636b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:18.945057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301af50-9a93-47ef-9862-2232b72636b5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7301af50-9a93-47ef-9862-2232b72636b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:51:18.959181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:18.960223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:18.961336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:51:18.961336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:51:18.961336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:51:18.961336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:51:18.961336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:51:18.961336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:51:18.961336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:18.961336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:18.961336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 616/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:18.966328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a669c-ff75-431f-83ca-99ffba62ce8c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:19.063543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a669c-ff75-431f-83ca-99ffba62ce8c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:19.085668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:19.097242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700c925-ce4f-404e-adc8-3f36218ef1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:19.097903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700c925-ce4f-404e-adc8-3f36218ef1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:19.098215 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700c925-ce4f-404e-adc8-3f36218ef1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:19.098492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700c925-ce4f-404e-adc8-3f36218ef1a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:19.099340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700c925-ce4f-404e-adc8-3f36218ef1a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:19.099649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700c925-ce4f-404e-adc8-3f36218ef1a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:19.101082 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700c925-ce4f-404e-adc8-3f36218ef1a7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:19.112428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 602/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:51:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:19.118418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301af50-9a93-47ef-9862-2232b72636b5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:19.219616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301af50-9a93-47ef-9862-2232b72636b5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:19.249592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:19.255341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e220c8-029b-49a9-a667-364e2f07bcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:19.255993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e220c8-029b-49a9-a667-364e2f07bcd2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:19.256273 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e220c8-029b-49a9-a667-364e2f07bcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:19.256544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e220c8-029b-49a9-a667-364e2f07bcd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:19.257367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e220c8-029b-49a9-a667-364e2f07bcd2 None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=bca7c1164d3e4833a10ffc37cbc6f092) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:19.257674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e220c8-029b-49a9-a667-364e2f07bcd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:19.259131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e220c8-029b-49a9-a667-364e2f07bcd2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:19.264584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 617/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:51:19 2022] GET /identity/v3/role_assignments?user.id=bca7c1164d3e4833a10ffc37cbc6f092&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:21.260834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ceee4-0897-4dcc-89c8-8fcbe558e183 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:21.261452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ceee4-0897-4dcc-89c8-8fcbe558e183 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:21.261452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ceee4-0897-4dcc-89c8-8fcbe558e183 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:21.263070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 603/1220] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:21.268573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5990a-e2cd-454c-ad76-54df1e06cd5c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:21.269070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5990a-e2cd-454c-ad76-54df1e06cd5c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:21.269278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5990a-e2cd-454c-ad76-54df1e06cd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:21.286971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ace5990a-e2cd-454c-ad76-54df1e06cd5c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:21.288150 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ace5990a-e2cd-454c-ad76-54df1e06cd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:21.355013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 618/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:21.378622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ceee4-0897-4dcc-89c8-8fcbe558e183 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:21.479097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ceee4-0897-4dcc-89c8-8fcbe558e183 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:21.513874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a223ea-507a-475f-a8d2-56cecf6c9ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:21.514612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a223ea-507a-475f-a8d2-56cecf6c9ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:21.514889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a223ea-507a-475f-a8d2-56cecf6c9ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:21.515156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a223ea-507a-475f-a8d2-56cecf6c9ac1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:21.524868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a223ea-507a-475f-a8d2-56cecf6c9ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-00a223ea-507a-475f-a8d2-56cecf6c9ac1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:51:21.533444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:21.538327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:21.539481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 604/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:51:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:21.541739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5990a-e2cd-454c-ad76-54df1e06cd5c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:21.638882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5990a-e2cd-454c-ad76-54df1e06cd5c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:21.662281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:21.673411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95774ef5-3062-480b-9bc3-70e830f26bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:21.674098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774ef5-3062-480b-9bc3-70e830f26bef None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:21.674383 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774ef5-3062-480b-9bc3-70e830f26bef None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:21.674660 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774ef5-3062-480b-9bc3-70e830f26bef None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:21.675404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95774ef5-3062-480b-9bc3-70e830f26bef None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:21.675710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95774ef5-3062-480b-9bc3-70e830f26bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:21.677078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95774ef5-3062-480b-9bc3-70e830f26bef None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:21.685145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 619/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:51:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:21.691029 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a223ea-507a-475f-a8d2-56cecf6c9ac1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:21.735360 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:21.790871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a223ea-507a-475f-a8d2-56cecf6c9ac1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:21.823770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88356bdc-c6e1-450e-9676-1494a05df9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:21.824545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88356bdc-c6e1-450e-9676-1494a05df9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:21.825412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88356bdc-c6e1-450e-9676-1494a05df9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:21.825692 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88356bdc-c6e1-450e-9676-1494a05df9b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:21.835668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88356bdc-c6e1-450e-9676-1494a05df9b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:21.837107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88356bdc-c6e1-450e-9676-1494a05df9b7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-88356bdc-c6e1-450e-9676-1494a05df9b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:21.854737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:21.855809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:21.857034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:21.857034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:21.857034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:21.857034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:21.857034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 605/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:21.861811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95774ef5-3062-480b-9bc3-70e830f26bef None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:21.959985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95774ef5-3062-480b-9bc3-70e830f26bef None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:21.992371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d11082-cdbb-4cc9-b418-c367e0b9a278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:21.993014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d11082-cdbb-4cc9-b418-c367e0b9a278 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:21.993292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d11082-cdbb-4cc9-b418-c367e0b9a278 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:21.993568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d11082-cdbb-4cc9-b418-c367e0b9a278 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:21.994301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d11082-cdbb-4cc9-b418-c367e0b9a278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:21.994603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d11082-cdbb-4cc9-b418-c367e0b9a278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:21.995957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d11082-cdbb-4cc9-b418-c367e0b9a278 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:22.007831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 620/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:22.014708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88356bdc-c6e1-450e-9676-1494a05df9b7 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:22.107127 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:22.113700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88356bdc-c6e1-450e-9676-1494a05df9b7 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:22.147455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266d6e6-79d4-4138-8325-2d5f719cbbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:22.148241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266d6e6-79d4-4138-8325-2d5f719cbbbc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:22.148525 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266d6e6-79d4-4138-8325-2d5f719cbbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:22.148801 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266d6e6-79d4-4138-8325-2d5f719cbbbc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:22.156590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266d6e6-79d4-4138-8325-2d5f719cbbbc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:22.158204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266d6e6-79d4-4138-8325-2d5f719cbbbc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-c266d6e6-79d4-4138-8325-2d5f719cbbbc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:51:22.173481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:22.174616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:22.175985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 606/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:22.180755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d11082-cdbb-4cc9-b418-c367e0b9a278 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:22.226893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:22.278590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d11082-cdbb-4cc9-b418-c367e0b9a278 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:22.313101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c69ec-1969-400c-8ef4-1ca51b15e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:22.313756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c69ec-1969-400c-8ef4-1ca51b15e10b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:22.314060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c69ec-1969-400c-8ef4-1ca51b15e10b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:22.314344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c69ec-1969-400c-8ef4-1ca51b15e10b None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:22.315172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c69ec-1969-400c-8ef4-1ca51b15e10b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:22.315501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c69ec-1969-400c-8ef4-1ca51b15e10b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:22.316951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c69ec-1969-400c-8ef4-1ca51b15e10b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:22.333617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 621/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 13 08:51:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:22.339621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266d6e6-79d4-4138-8325-2d5f719cbbbc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:22.437731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266d6e6-79d4-4138-8325-2d5f719cbbbc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:22.470519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa15c1a-7a24-48bb-a0cd-8fbfd01c70a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:22.471292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa15c1a-7a24-48bb-a0cd-8fbfd01c70a3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:22.471571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa15c1a-7a24-48bb-a0cd-8fbfd01c70a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:22.471843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa15c1a-7a24-48bb-a0cd-8fbfd01c70a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:22.482466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa15c1a-7a24-48bb-a0cd-8fbfd01c70a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:22.483885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa15c1a-7a24-48bb-a0cd-8fbfd01c70a3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-2fa15c1a-7a24-48bb-a0cd-8fbfd01c70a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:22.501875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:22.502934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:22.504241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:22.504241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:22.504241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:22.504241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:22.504241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 607/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:22.508751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c69ec-1969-400c-8ef4-1ca51b15e10b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:22.608845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c69ec-1969-400c-8ef4-1ca51b15e10b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:22.611208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:22.642583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc179d3-4975-4c7c-8f32-6f49460a7f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:22.643229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc179d3-4975-4c7c-8f32-6f49460a7f93 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:22.643505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc179d3-4975-4c7c-8f32-6f49460a7f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:22.643775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc179d3-4975-4c7c-8f32-6f49460a7f93 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:22.644479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc179d3-4975-4c7c-8f32-6f49460a7f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:22.644788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc179d3-4975-4c7c-8f32-6f49460a7f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:22.646145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc179d3-4975-4c7c-8f32-6f49460a7f93 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:22.656896 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 622/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:22.662556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa15c1a-7a24-48bb-a0cd-8fbfd01c70a3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:22.672159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:22.761217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa15c1a-7a24-48bb-a0cd-8fbfd01c70a3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:22.795725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f67b9-f8e0-4cdd-bd3b-b107e137dcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:22.796501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f67b9-f8e0-4cdd-bd3b-b107e137dcac None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:22.796774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f67b9-f8e0-4cdd-bd3b-b107e137dcac None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:22.797038 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f67b9-f8e0-4cdd-bd3b-b107e137dcac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:22.803580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f67b9-f8e0-4cdd-bd3b-b107e137dcac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:22.805017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f67b9-f8e0-4cdd-bd3b-b107e137dcac None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-444f67b9-f8e0-4cdd-bd3b-b107e137dcac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:51:22.819269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:22.820267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:22.821199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:51:22.821199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:51:22.821199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:51:22.821199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:51:22.821199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:51:22.821199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:51:22.821199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:22.821199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:22.821199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 608/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:22.826942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc179d3-4975-4c7c-8f32-6f49460a7f93 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:22.925115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc179d3-4975-4c7c-8f32-6f49460a7f93 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:22.958416 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadfff6-d3d2-4b83-9c2a-b6b33043854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:22.959062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadfff6-d3d2-4b83-9c2a-b6b33043854a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:22.959347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadfff6-d3d2-4b83-9c2a-b6b33043854a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:22.959626 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadfff6-d3d2-4b83-9c2a-b6b33043854a None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:22.960558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadfff6-d3d2-4b83-9c2a-b6b33043854a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:22.960867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadfff6-d3d2-4b83-9c2a-b6b33043854a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:22.962337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadfff6-d3d2-4b83-9c2a-b6b33043854a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:22.974867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 623/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:22.980799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f67b9-f8e0-4cdd-bd3b-b107e137dcac None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:23.045793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:23.080181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f67b9-f8e0-4cdd-bd3b-b107e137dcac None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:23.112966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024968b5-5460-466a-91dd-c903d7011613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:23.113839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024968b5-5460-466a-91dd-c903d7011613 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:23.114152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024968b5-5460-466a-91dd-c903d7011613 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:23.114424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024968b5-5460-466a-91dd-c903d7011613 None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/bca7c1164d3e4833a10ffc37cbc6f092/roles/3ec600afe034440d873fe4834644b63b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:23.128962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024968b5-5460-466a-91dd-c903d7011613 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=bca7c1164d3e4833a10ffc37cbc6f092, role_id=3ec600afe034440d873fe4834644b63b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:23.132319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024968b5-5460-466a-91dd-c903d7011613 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:23.233483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 609/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:51:22 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/bca7c1164d3e4833a10ffc37cbc6f092/roles/3ec600afe034440d873fe4834644b63b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:51:25.235703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d14b9-3f4b-4874-9408-dd54d7f328f3 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:25.236234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d14b9-3f4b-4874-9408-dd54d7f328f3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:25.236234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d14b9-3f4b-4874-9408-dd54d7f328f3 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:25.237685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 624/1233] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:25.242867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da80a19-f169-4244-9b87-bb30cb6b54bc None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:25.245484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da80a19-f169-4244-9b87-bb30cb6b54bc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:25.245484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da80a19-f169-4244-9b87-bb30cb6b54bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:25.260522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7da80a19-f169-4244-9b87-bb30cb6b54bc None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:25.261700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7da80a19-f169-4244-9b87-bb30cb6b54bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:25.328471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 610/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:25.353174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d14b9-3f4b-4874-9408-dd54d7f328f3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:25.378776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:25.457612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d14b9-3f4b-4874-9408-dd54d7f328f3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:25.492485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85609313-0d00-4153-aa0c-31ac55dfff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:25.493198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609313-0d00-4153-aa0c-31ac55dfff9b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:25.493477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609313-0d00-4153-aa0c-31ac55dfff9b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:25.493747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609313-0d00-4153-aa0c-31ac55dfff9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:25.504093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85609313-0d00-4153-aa0c-31ac55dfff9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-85609313-0d00-4153-aa0c-31ac55dfff9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:51:25.511673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:25.512874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:25.513782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 625/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 13 08:51:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:25.519411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da80a19-f169-4244-9b87-bb30cb6b54bc None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:25.565363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:25.617152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da80a19-f169-4244-9b87-bb30cb6b54bc None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:25.651154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b170a6-77a3-46e3-977a-a389c0ec204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:25.651814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b170a6-77a3-46e3-977a-a389c0ec204c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:25.652138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b170a6-77a3-46e3-977a-a389c0ec204c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:25.652409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b170a6-77a3-46e3-977a-a389c0ec204c None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:25.654294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b170a6-77a3-46e3-977a-a389c0ec204c None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:25.654602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b170a6-77a3-46e3-977a-a389c0ec204c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:25.655991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b170a6-77a3-46e3-977a-a389c0ec204c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:25.664093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 611/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 13 08:51:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:25.670289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85609313-0d00-4153-aa0c-31ac55dfff9b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:25.753943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:25.771240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85609313-0d00-4153-aa0c-31ac55dfff9b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:25.807924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7b008-3c12-40cb-aeaf-abf3455ad6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:25.808770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7b008-3c12-40cb-aeaf-abf3455ad6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:25.809125 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7b008-3c12-40cb-aeaf-abf3455ad6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:25.809526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7b008-3c12-40cb-aeaf-abf3455ad6d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:25.820221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7b008-3c12-40cb-aeaf-abf3455ad6d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:25.821664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7b008-3c12-40cb-aeaf-abf3455ad6d7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a1b7b008-3c12-40cb-aeaf-abf3455ad6d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:25.839013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:25.840168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:25.841993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:25.841993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:25.841993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:25.841993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:25.841993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 626/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:25.846544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b170a6-77a3-46e3-977a-a389c0ec204c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:25.944117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b170a6-77a3-46e3-977a-a389c0ec204c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:25.953373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:25.977893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a85f07-fbda-458b-a7d4-ef8c353ac7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:25.978579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a85f07-fbda-458b-a7d4-ef8c353ac7ff None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:25.978862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a85f07-fbda-458b-a7d4-ef8c353ac7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:25.979135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a85f07-fbda-458b-a7d4-ef8c353ac7ff None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:25.979841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a85f07-fbda-458b-a7d4-ef8c353ac7ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:25.980486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a85f07-fbda-458b-a7d4-ef8c353ac7ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:25.981741 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a85f07-fbda-458b-a7d4-ef8c353ac7ff None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:25.993449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 612/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:25.999620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7b008-3c12-40cb-aeaf-abf3455ad6d7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:26.100158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7b008-3c12-40cb-aeaf-abf3455ad6d7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:26.132798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b17ea8-c087-41fb-931f-09a29291ae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:26.133584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b17ea8-c087-41fb-931f-09a29291ae44 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:26.133862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b17ea8-c087-41fb-931f-09a29291ae44 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:26.134194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b17ea8-c087-41fb-931f-09a29291ae44 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:26.136819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:26.142627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b17ea8-c087-41fb-931f-09a29291ae44 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:26.144058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b17ea8-c087-41fb-931f-09a29291ae44 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a3b17ea8-c087-41fb-931f-09a29291ae44 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:51:26.158566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:26.159685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:26.160794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 627/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:26.165784 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a85f07-fbda-458b-a7d4-ef8c353ac7ff None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:26.265052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a85f07-fbda-458b-a7d4-ef8c353ac7ff None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:26.299141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea782df-5bad-4fcc-b1b6-87bb6c2ff73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:26.299827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea782df-5bad-4fcc-b1b6-87bb6c2ff73f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:26.300100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea782df-5bad-4fcc-b1b6-87bb6c2ff73f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:26.300366 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea782df-5bad-4fcc-b1b6-87bb6c2ff73f None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:26.301187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea782df-5bad-4fcc-b1b6-87bb6c2ff73f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:26.301485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea782df-5bad-4fcc-b1b6-87bb6c2ff73f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:26.302921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea782df-5bad-4fcc-b1b6-87bb6c2ff73f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:26.320364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 613/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 13 08:51:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:26.326866 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b17ea8-c087-41fb-931f-09a29291ae44 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:26.429221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b17ea8-c087-41fb-931f-09a29291ae44 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:26.462294 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa6d90-1f64-4139-97b2-386d56ad5c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:26.462933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa6d90-1f64-4139-97b2-386d56ad5c92 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:26.463215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa6d90-1f64-4139-97b2-386d56ad5c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:26.463490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa6d90-1f64-4139-97b2-386d56ad5c92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:26.524343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:26.530570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa6d90-1f64-4139-97b2-386d56ad5c92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:26.532023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa6d90-1f64-4139-97b2-386d56ad5c92 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:26.628652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 628/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:51:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:51:26.634474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea782df-5bad-4fcc-b1b6-87bb6c2ff73f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:26.662138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:26.734750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea782df-5bad-4fcc-b1b6-87bb6c2ff73f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:26.768113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f92c9-737e-4387-ae78-18f6ee7c838f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:26.768911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f92c9-737e-4387-ae78-18f6ee7c838f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:26.769189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f92c9-737e-4387-ae78-18f6ee7c838f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:26.769457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f92c9-737e-4387-ae78-18f6ee7c838f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:26.780397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f92c9-737e-4387-ae78-18f6ee7c838f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:26.786162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f92c9-737e-4387-ae78-18f6ee7c838f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ec3f92c9-737e-4387-ae78-18f6ee7c838f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:26.800114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:26.801239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:26.802205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:26.802205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:26.802205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:26.802205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:26.802205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 614/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:26.807031 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa6d90-1f64-4139-97b2-386d56ad5c92 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:26.897772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:26.903837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa6d90-1f64-4139-97b2-386d56ad5c92 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:26.936141 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05fb20-4b45-4159-b991-915a7d9225d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:26.936784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05fb20-4b45-4159-b991-915a7d9225d6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:26.937060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05fb20-4b45-4159-b991-915a7d9225d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:26.937330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05fb20-4b45-4159-b991-915a7d9225d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:26.938043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05fb20-4b45-4159-b991-915a7d9225d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:26.938407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05fb20-4b45-4159-b991-915a7d9225d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:26.939788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05fb20-4b45-4159-b991-915a7d9225d6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:26.950632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 629/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:26.956254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f92c9-737e-4387-ae78-18f6ee7c838f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:27.034083 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:27.055531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f92c9-737e-4387-ae78-18f6ee7c838f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:27.088327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddb56f-bc48-491b-8979-bef607b50fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:27.089133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddb56f-bc48-491b-8979-bef607b50fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:27.089415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddb56f-bc48-491b-8979-bef607b50fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:27.089689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddb56f-bc48-491b-8979-bef607b50fd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:27.096749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddb56f-bc48-491b-8979-bef607b50fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:27.098223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddb56f-bc48-491b-8979-bef607b50fd3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d3ddb56f-bc48-491b-8979-bef607b50fd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:51:27.112442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:27.113538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:27.114693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:51:27.114693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:51:27.114693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:51:27.114693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:51:27.114693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:51:27.114693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:51:27.114693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:27.114693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:27.114693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 615/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:27.119841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05fb20-4b45-4159-b991-915a7d9225d6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:27.220658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05fb20-4b45-4159-b991-915a7d9225d6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:27.252487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33382fb7-9974-45d2-90d0-3125af3acf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:27.253124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33382fb7-9974-45d2-90d0-3125af3acf17 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:27.253401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33382fb7-9974-45d2-90d0-3125af3acf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:27.253677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33382fb7-9974-45d2-90d0-3125af3acf17 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:27.254483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33382fb7-9974-45d2-90d0-3125af3acf17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:27.254791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33382fb7-9974-45d2-90d0-3125af3acf17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:27.256234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33382fb7-9974-45d2-90d0-3125af3acf17 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:27.268778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 630/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:27.274783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddb56f-bc48-491b-8979-bef607b50fd3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:27.379231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddb56f-bc48-491b-8979-bef607b50fd3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:27.412063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16980cfb-131d-499d-b62b-323b5f33c834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:27.412732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16980cfb-131d-499d-b62b-323b5f33c834 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:27.413016 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16980cfb-131d-499d-b62b-323b5f33c834 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:27.413291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16980cfb-131d-499d-b62b-323b5f33c834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:27.414443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16980cfb-131d-499d-b62b-323b5f33c834 None admin] RBAC: Adding query filter params (role.id=3ec600afe034440d873fe4834644b63b, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=bca7c1164d3e4833a10ffc37cbc6f092) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:27.414755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16980cfb-131d-499d-b62b-323b5f33c834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:27.416175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16980cfb-131d-499d-b62b-323b5f33c834 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:27.422472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 616/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:51:27 2022] GET /identity/v3/role_assignments?user.id=bca7c1164d3e4833a10ffc37cbc6f092&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=3ec600afe034440d873fe4834644b63b => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:29.505239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2954e7c-1dac-4b4b-a7e8-fc30d5ef7b30 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:29.505824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2954e7c-1dac-4b4b-a7e8-fc30d5ef7b30 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:29.505824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2954e7c-1dac-4b4b-a7e8-fc30d5ef7b30 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:29.507214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 631/1247] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:29.512306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d1cd8-93e7-4a20-bca8-9dc522c0fd69 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:29.512890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d1cd8-93e7-4a20-bca8-9dc522c0fd69 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:29.512890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d1cd8-93e7-4a20-bca8-9dc522c0fd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:29.518472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:29.530652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-281d1cd8-93e7-4a20-bca8-9dc522c0fd69 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:29.531819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-281d1cd8-93e7-4a20-bca8-9dc522c0fd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:29.596283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 617/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:29.620305 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2954e7c-1dac-4b4b-a7e8-fc30d5ef7b30 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:29.639323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:29.721805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2954e7c-1dac-4b4b-a7e8-fc30d5ef7b30 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:29.754390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f62b7c-ad6b-4a4a-a83c-fecc43244c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:29.755167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f62b7c-ad6b-4a4a-a83c-fecc43244c89 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:29.755447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f62b7c-ad6b-4a4a-a83c-fecc43244c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:29.755723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f62b7c-ad6b-4a4a-a83c-fecc43244c89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:29.766219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f62b7c-ad6b-4a4a-a83c-fecc43244c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-29f62b7c-ad6b-4a4a-a83c-fecc43244c89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:51:29.773748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:29.774891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:29.775924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 632/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:51:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:51:29.781192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d1cd8-93e7-4a20-bca8-9dc522c0fd69 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:29.879990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d1cd8-93e7-4a20-bca8-9dc522c0fd69 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:29.909315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:29.914426 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e64fe-f88c-4397-9278-e11fec9169dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:29.915087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e64fe-f88c-4397-9278-e11fec9169dd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:29.915369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e64fe-f88c-4397-9278-e11fec9169dd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:29.915642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e64fe-f88c-4397-9278-e11fec9169dd None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:29.916375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e64fe-f88c-4397-9278-e11fec9169dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:29.916678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e64fe-f88c-4397-9278-e11fec9169dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:29.918567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e64fe-f88c-4397-9278-e11fec9169dd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:29.926428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 618/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:51:29.932313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f62b7c-ad6b-4a4a-a83c-fecc43244c89 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:30.010657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:30.033140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f62b7c-ad6b-4a4a-a83c-fecc43244c89 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:30.066372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb414d7e-a4e4-4b82-a85e-fe61c1208e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:30.067148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb414d7e-a4e4-4b82-a85e-fe61c1208e7b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:30.067426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb414d7e-a4e4-4b82-a85e-fe61c1208e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:30.067697 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb414d7e-a4e4-4b82-a85e-fe61c1208e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:30.078141 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb414d7e-a4e4-4b82-a85e-fe61c1208e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:30.079576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb414d7e-a4e4-4b82-a85e-fe61c1208e7b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-bb414d7e-a4e4-4b82-a85e-fe61c1208e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:30.096277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:30.097364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:30.098540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:30.098540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:30.098540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:30.098540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:30.098540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 633/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:30.103415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e64fe-f88c-4397-9278-e11fec9169dd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:30.203218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e64fe-f88c-4397-9278-e11fec9169dd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:30.237722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741485ad-2d94-46fc-a830-6dfca4e1b8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:30.238401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741485ad-2d94-46fc-a830-6dfca4e1b8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:30.238683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741485ad-2d94-46fc-a830-6dfca4e1b8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:30.238954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741485ad-2d94-46fc-a830-6dfca4e1b8c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:30.239657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741485ad-2d94-46fc-a830-6dfca4e1b8c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:30.239985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741485ad-2d94-46fc-a830-6dfca4e1b8c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:30.241340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741485ad-2d94-46fc-a830-6dfca4e1b8c4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:30.252876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 619/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:30.258914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb414d7e-a4e4-4b82-a85e-fe61c1208e7b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:30.359623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb414d7e-a4e4-4b82-a85e-fe61c1208e7b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:30.384985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:30.392920 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765da096-fbef-42be-9df6-d45a1e74d721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:30.393734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765da096-fbef-42be-9df6-d45a1e74d721 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:30.394016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765da096-fbef-42be-9df6-d45a1e74d721 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:30.394315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765da096-fbef-42be-9df6-d45a1e74d721 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:30.401601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765da096-fbef-42be-9df6-d45a1e74d721 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:30.403706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765da096-fbef-42be-9df6-d45a1e74d721 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-765da096-fbef-42be-9df6-d45a1e74d721 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:51:30.418705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:30.420082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:30.421225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 634/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:30.426330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741485ad-2d94-46fc-a830-6dfca4e1b8c4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:30.480069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:30.523317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741485ad-2d94-46fc-a830-6dfca4e1b8c4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:30.555432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fc903-b429-4240-bec8-a547c1b3e08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:30.556080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fc903-b429-4240-bec8-a547c1b3e08a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:30.556354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fc903-b429-4240-bec8-a547c1b3e08a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:30.556622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fc903-b429-4240-bec8-a547c1b3e08a None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:30.557437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fc903-b429-4240-bec8-a547c1b3e08a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:30.557739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fc903-b429-4240-bec8-a547c1b3e08a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:30.559398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fc903-b429-4240-bec8-a547c1b3e08a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:30.576974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 620/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 13 08:51:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:30.582668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765da096-fbef-42be-9df6-d45a1e74d721 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:30.680616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765da096-fbef-42be-9df6-d45a1e74d721 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:30.713786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c264a6-ff24-4667-a628-3694a8f4d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:30.714456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c264a6-ff24-4667-a628-3694a8f4d29e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:30.714737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c264a6-ff24-4667-a628-3694a8f4d29e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:30.715642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c264a6-ff24-4667-a628-3694a8f4d29e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:30.773160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:30.785070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c264a6-ff24-4667-a628-3694a8f4d29e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:30.786496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c264a6-ff24-4667-a628-3694a8f4d29e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:30.884908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 635/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:51:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:51:30.890898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fc903-b429-4240-bec8-a547c1b3e08a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:30.992458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fc903-b429-4240-bec8-a547c1b3e08a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:31.005849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:31.026372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbefac-3d57-490f-bdae-23fc2ff59cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:31.027095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbefac-3d57-490f-bdae-23fc2ff59cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:31.027853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbefac-3d57-490f-bdae-23fc2ff59cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:31.028129 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbefac-3d57-490f-bdae-23fc2ff59cf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:31.037905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbefac-3d57-490f-bdae-23fc2ff59cf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:31.039611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbefac-3d57-490f-bdae-23fc2ff59cf6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-36fbefac-3d57-490f-bdae-23fc2ff59cf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:31.058021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:31.059397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:31.060460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:31.060460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:31.060460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:31.060460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:31.060460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 621/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:31.065062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c264a6-ff24-4667-a628-3694a8f4d29e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:31.152232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:31.165262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c264a6-ff24-4667-a628-3694a8f4d29e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:31.201523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4676cb-dc75-4e74-b952-ab8e9d6e7742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:31.202212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4676cb-dc75-4e74-b952-ab8e9d6e7742 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:31.202625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4676cb-dc75-4e74-b952-ab8e9d6e7742 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:31.202907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4676cb-dc75-4e74-b952-ab8e9d6e7742 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:31.203625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4676cb-dc75-4e74-b952-ab8e9d6e7742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:31.203931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4676cb-dc75-4e74-b952-ab8e9d6e7742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:31.205300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4676cb-dc75-4e74-b952-ab8e9d6e7742 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:31.217473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 636/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:31.223132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbefac-3d57-490f-bdae-23fc2ff59cf6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:31.319668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbefac-3d57-490f-bdae-23fc2ff59cf6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:31.353950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039101fe-61c0-4945-b79c-53ae14b552ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:31.354765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039101fe-61c0-4945-b79c-53ae14b552ee None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:31.355044 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039101fe-61c0-4945-b79c-53ae14b552ee None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:31.355314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039101fe-61c0-4945-b79c-53ae14b552ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:31.362245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039101fe-61c0-4945-b79c-53ae14b552ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:31.363689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039101fe-61c0-4945-b79c-53ae14b552ee None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-039101fe-61c0-4945-b79c-53ae14b552ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:51:31.377535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:31.378637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:31.379972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:51:31.379972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:51:31.379972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:51:31.379972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:51:31.379972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:51:31.379972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:51:31.379972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:31.379972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:31.379972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 622/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:51:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:31.384465 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4676cb-dc75-4e74-b952-ab8e9d6e7742 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:31.488116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4676cb-dc75-4e74-b952-ab8e9d6e7742 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:31.525968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32739da-b02a-4d0e-be84-430fd76b101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:31.526657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32739da-b02a-4d0e-be84-430fd76b101b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:31.526939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32739da-b02a-4d0e-be84-430fd76b101b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:31.527215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32739da-b02a-4d0e-be84-430fd76b101b None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:31.528915 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32739da-b02a-4d0e-be84-430fd76b101b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:31.529224 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32739da-b02a-4d0e-be84-430fd76b101b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:31.530729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32739da-b02a-4d0e-be84-430fd76b101b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:31.545620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 637/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:51:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:31.551474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039101fe-61c0-4945-b79c-53ae14b552ee None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:31.578202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:31.657470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039101fe-61c0-4945-b79c-53ae14b552ee None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:31.691174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3576139-4305-403c-97cf-b6f63a87099f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:31.691809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576139-4305-403c-97cf-b6f63a87099f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:31.692083 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576139-4305-403c-97cf-b6f63a87099f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:31.692352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576139-4305-403c-97cf-b6f63a87099f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:31.693158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3576139-4305-403c-97cf-b6f63a87099f None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=bca7c1164d3e4833a10ffc37cbc6f092) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:31.693459 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3576139-4305-403c-97cf-b6f63a87099f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:31.694955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3576139-4305-403c-97cf-b6f63a87099f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:31.701056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 623/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:51:31 2022] GET /identity/v3/role_assignments?user.id=bca7c1164d3e4833a10ffc37cbc6f092&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:33.694421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7135b-0fe5-441a-89e6-bba3a73a0779 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:33.694994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7135b-0fe5-441a-89e6-bba3a73a0779 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:33.694994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7135b-0fe5-441a-89e6-bba3a73a0779 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:33.696326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 638/1261] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:33.701565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298d887-3e12-47f2-bccb-7644818f7d9f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:33.702109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298d887-3e12-47f2-bccb-7644818f7d9f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:33.702254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298d887-3e12-47f2-bccb-7644818f7d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:33.718482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3298d887-3e12-47f2-bccb-7644818f7d9f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:33.722240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3298d887-3e12-47f2-bccb-7644818f7d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:33.785476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 624/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:33.808276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7135b-0fe5-441a-89e6-bba3a73a0779 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:33.819816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:33.909711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7135b-0fe5-441a-89e6-bba3a73a0779 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:33.944398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0dfd38-b95d-4a0d-b75b-22b149153bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:33.945243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dfd38-b95d-4a0d-b75b-22b149153bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:33.945466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dfd38-b95d-4a0d-b75b-22b149153bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:33.945740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0dfd38-b95d-4a0d-b75b-22b149153bb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:33.955831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0dfd38-b95d-4a0d-b75b-22b149153bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2c0dfd38-b95d-4a0d-b75b-22b149153bb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:51:33.963407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:33.964506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:33.965952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 639/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:33.971200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298d887-3e12-47f2-bccb-7644818f7d9f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:33.989702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:34.066699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298d887-3e12-47f2-bccb-7644818f7d9f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:34.100509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad49ee3-8730-4987-a93e-ec87a6615cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:34.101169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad49ee3-8730-4987-a93e-ec87a6615cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:34.101454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad49ee3-8730-4987-a93e-ec87a6615cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:34.101727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad49ee3-8730-4987-a93e-ec87a6615cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:34.102498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad49ee3-8730-4987-a93e-ec87a6615cd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:34.102804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad49ee3-8730-4987-a93e-ec87a6615cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:34.104157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad49ee3-8730-4987-a93e-ec87a6615cd3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:34.112795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 625/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:51:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:34.118875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0dfd38-b95d-4a0d-b75b-22b149153bb1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:34.187013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:34.220423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0dfd38-b95d-4a0d-b75b-22b149153bb1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:34.254088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e999c4-db32-4f23-8472-3d9d4f10f97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:34.254878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e999c4-db32-4f23-8472-3d9d4f10f97c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:34.255161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e999c4-db32-4f23-8472-3d9d4f10f97c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:34.255436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e999c4-db32-4f23-8472-3d9d4f10f97c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:34.266792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e999c4-db32-4f23-8472-3d9d4f10f97c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:34.268168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e999c4-db32-4f23-8472-3d9d4f10f97c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-79e999c4-db32-4f23-8472-3d9d4f10f97c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:34.285814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:34.286959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:34.287996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:34.287996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:34.287996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:34.287996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:34.287996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 640/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:34.293329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad49ee3-8730-4987-a93e-ec87a6615cd3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:34.375781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:34.395247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad49ee3-8730-4987-a93e-ec87a6615cd3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:34.429614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe2bcf-b016-456e-ba15-bfc1cb74b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:34.430301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe2bcf-b016-456e-ba15-bfc1cb74b382 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:34.430586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe2bcf-b016-456e-ba15-bfc1cb74b382 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:34.430865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe2bcf-b016-456e-ba15-bfc1cb74b382 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:34.431585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe2bcf-b016-456e-ba15-bfc1cb74b382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:34.431894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe2bcf-b016-456e-ba15-bfc1cb74b382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:34.433266 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe2bcf-b016-456e-ba15-bfc1cb74b382 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:34.444740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 626/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:34.451086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e999c4-db32-4f23-8472-3d9d4f10f97c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:34.552805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e999c4-db32-4f23-8472-3d9d4f10f97c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:34.573312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:34.587505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64d686-7c7c-4511-9f93-c14a74432276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:34.588293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64d686-7c7c-4511-9f93-c14a74432276 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:34.588576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64d686-7c7c-4511-9f93-c14a74432276 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:34.588850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64d686-7c7c-4511-9f93-c14a74432276 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:34.596849 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64d686-7c7c-4511-9f93-c14a74432276 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:34.598336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64d686-7c7c-4511-9f93-c14a74432276 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-cd64d686-7c7c-4511-9f93-c14a74432276 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:51:34.613195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:34.614379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:34.615535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 641/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:51:34.620259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe2bcf-b016-456e-ba15-bfc1cb74b382 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:34.722083 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe2bcf-b016-456e-ba15-bfc1cb74b382 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:34.755209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a09c60-acdc-451d-885d-ba9bd58a0934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:34.755859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a09c60-acdc-451d-885d-ba9bd58a0934 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:34.756136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a09c60-acdc-451d-885d-ba9bd58a0934 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:34.756406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a09c60-acdc-451d-885d-ba9bd58a0934 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:34.757226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a09c60-acdc-451d-885d-ba9bd58a0934 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:34.757528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a09c60-acdc-451d-885d-ba9bd58a0934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:34.759060 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a09c60-acdc-451d-885d-ba9bd58a0934 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:34.768753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:34.776518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 627/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 13 08:51:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:51:34.782763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64d686-7c7c-4511-9f93-c14a74432276 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:34.884388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64d686-7c7c-4511-9f93-c14a74432276 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:34.917662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0ce95-65f8-49d6-902c-3248ce52d68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:34.919068 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0ce95-65f8-49d6-902c-3248ce52d68b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:34.919351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0ce95-65f8-49d6-902c-3248ce52d68b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:34.919625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0ce95-65f8-49d6-902c-3248ce52d68b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:34.930062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0ce95-65f8-49d6-902c-3248ce52d68b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:34.931533 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0ce95-65f8-49d6-902c-3248ce52d68b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5bb0ce95-65f8-49d6-902c-3248ce52d68b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:34.948408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:34.949702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:34.950842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:34.950842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:34.950842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:34.950842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:34.950842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 642/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:34.955689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a09c60-acdc-451d-885d-ba9bd58a0934 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:35.057170 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a09c60-acdc-451d-885d-ba9bd58a0934 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:35.092239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fb6c0-9235-413b-8d62-23579785a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:35.092894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fb6c0-9235-413b-8d62-23579785a6bc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:35.093175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fb6c0-9235-413b-8d62-23579785a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:35.093451 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fb6c0-9235-413b-8d62-23579785a6bc None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:35.094243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fb6c0-9235-413b-8d62-23579785a6bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:35.094543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fb6c0-9235-413b-8d62-23579785a6bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:35.095897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7fb6c0-9235-413b-8d62-23579785a6bc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:35.106984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 628/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:35.112754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0ce95-65f8-49d6-902c-3248ce52d68b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:35.145403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:35.213111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0ce95-65f8-49d6-902c-3248ce52d68b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:35.246425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07830dfe-9eb2-47f9-ab99-571d7e3612cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:35.247225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07830dfe-9eb2-47f9-ab99-571d7e3612cb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:35.247509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07830dfe-9eb2-47f9-ab99-571d7e3612cb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:35.247788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07830dfe-9eb2-47f9-ab99-571d7e3612cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:35.254436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07830dfe-9eb2-47f9-ab99-571d7e3612cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:35.255874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07830dfe-9eb2-47f9-ab99-571d7e3612cb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-07830dfe-9eb2-47f9-ab99-571d7e3612cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:51:35.270671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:35.271657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:35.273032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:51:35.273032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:51:35.273032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:51:35.273032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:51:35.273032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:51:35.273032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:51:35.273032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:35.273032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:35.273032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 643/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:51:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:35.278532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fb6c0-9235-413b-8d62-23579785a6bc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:35.336086 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:35.377776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fb6c0-9235-413b-8d62-23579785a6bc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:35.411641 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360a5df-3f35-43fa-9450-7b9756c5bdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:35.412213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360a5df-3f35-43fa-9450-7b9756c5bdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:35.412497 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360a5df-3f35-43fa-9450-7b9756c5bdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:35.412769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360a5df-3f35-43fa-9450-7b9756c5bdd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:35.413804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360a5df-3f35-43fa-9450-7b9756c5bdd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:35.414143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360a5df-3f35-43fa-9450-7b9756c5bdd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:35.415565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360a5df-3f35-43fa-9450-7b9756c5bdd5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:35.428419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 629/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:35.435112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07830dfe-9eb2-47f9-ab99-571d7e3612cb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:35.525189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:35.533822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07830dfe-9eb2-47f9-ab99-571d7e3612cb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:35.566934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222c99b-8b6e-4bb8-9b10-f56080342f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:35.567840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222c99b-8b6e-4bb8-9b10-f56080342f48 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:35.568124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222c99b-8b6e-4bb8-9b10-f56080342f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:35.568403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222c99b-8b6e-4bb8-9b10-f56080342f48 None admin] PATH_INFO: `/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/bca7c1164d3e4833a10ffc37cbc6f092/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:35.570389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222c99b-8b6e-4bb8-9b10-f56080342f48 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa56bd5dbb81408c8fe362c5a1c02105, user_id=bca7c1164d3e4833a10ffc37cbc6f092, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:35.571827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222c99b-8b6e-4bb8-9b10-f56080342f48 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:35.683318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 644/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 13 08:51:35 2022] PUT /identity/v3/projects/aa56bd5dbb81408c8fe362c5a1c02105/users/bca7c1164d3e4833a10ffc37cbc6f092/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 08:51:37.697692 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b4a8-3a90-42dd-bd11-96124fb3b638 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:37.698246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b4a8-3a90-42dd-bd11-96124fb3b638 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:37.698355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b4a8-3a90-42dd-bd11-96124fb3b638 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:37.700238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 630/1274] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:37.705512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b03f3c-dc77-44e8-9d91-9556337d5c21 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:37.705911 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b03f3c-dc77-44e8-9d91-9556337d5c21 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:37.706087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b03f3c-dc77-44e8-9d91-9556337d5c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:37.722249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-02b03f3c-dc77-44e8-9d91-9556337d5c21 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:37.723412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-02b03f3c-dc77-44e8-9d91-9556337d5c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:37.789851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 645/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:37.814566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912b4a8-3a90-42dd-bd11-96124fb3b638 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:37.915490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912b4a8-3a90-42dd-bd11-96124fb3b638 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:37.932998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:37.950349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd90f38-0175-426c-a7d2-d9dfd981977e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:37.951068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd90f38-0175-426c-a7d2-d9dfd981977e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:37.951347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd90f38-0175-426c-a7d2-d9dfd981977e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:37.951617 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd90f38-0175-426c-a7d2-d9dfd981977e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:37.961182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd90f38-0175-426c-a7d2-d9dfd981977e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3fd90f38-0175-426c-a7d2-d9dfd981977e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 08:51:37.969169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:37.970187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:37.971049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 631/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 13 08:51:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:37.976525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b03f3c-dc77-44e8-9d91-9556337d5c21 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:38.056140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:38.073794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b03f3c-dc77-44e8-9d91-9556337d5c21 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:38.107011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1f218-13fb-4cbd-94b1-a557a9bde5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:38.107669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1f218-13fb-4cbd-94b1-a557a9bde5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:38.107948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1f218-13fb-4cbd-94b1-a557a9bde5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:38.108224 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1f218-13fb-4cbd-94b1-a557a9bde5f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:38.108959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1f218-13fb-4cbd-94b1-a557a9bde5f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:38.109262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1f218-13fb-4cbd-94b1-a557a9bde5f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:38.110772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1f218-13fb-4cbd-94b1-a557a9bde5f9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:38.119000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 646/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 13 08:51:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:38.125360 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd90f38-0175-426c-a7d2-d9dfd981977e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:38.224562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd90f38-0175-426c-a7d2-d9dfd981977e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:38.257608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceaaed-9561-40f7-aa12-ad3eea35d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:38.258415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceaaed-9561-40f7-aa12-ad3eea35d3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:38.258692 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceaaed-9561-40f7-aa12-ad3eea35d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:38.258970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceaaed-9561-40f7-aa12-ad3eea35d3b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:38.269380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bceaaed-9561-40f7-aa12-ad3eea35d3b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:38.270857 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bceaaed-9561-40f7-aa12-ad3eea35d3b1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3bceaaed-9561-40f7-aa12-ad3eea35d3b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:38.287537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:38.288870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:38.290233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:38.290233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:38.290233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:38.290233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:38.290233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 632/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:38.295366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1f218-13fb-4cbd-94b1-a557a9bde5f9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:38.396315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1f218-13fb-4cbd-94b1-a557a9bde5f9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:38.431131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb18a9-608d-423d-afcf-0edbac2ef8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:38.431780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb18a9-608d-423d-afcf-0edbac2ef8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:38.432061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb18a9-608d-423d-afcf-0edbac2ef8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:38.432342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb18a9-608d-423d-afcf-0edbac2ef8e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:38.433059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb18a9-608d-423d-afcf-0edbac2ef8e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:38.433365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb18a9-608d-423d-afcf-0edbac2ef8e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:38.434760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb18a9-608d-423d-afcf-0edbac2ef8e3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:38.446271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 647/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:51:38.452004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceaaed-9561-40f7-aa12-ad3eea35d3b1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:38.483628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:38.551130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceaaed-9561-40f7-aa12-ad3eea35d3b1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:38.584680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919f44c-7268-44ac-88ae-735a30cd897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:38.585470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919f44c-7268-44ac-88ae-735a30cd897a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:38.585751 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919f44c-7268-44ac-88ae-735a30cd897a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:38.586024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919f44c-7268-44ac-88ae-735a30cd897a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:38.593582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919f44c-7268-44ac-88ae-735a30cd897a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:38.595066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919f44c-7268-44ac-88ae-735a30cd897a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-6919f44c-7268-44ac-88ae-735a30cd897a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 08:51:38.610247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:38.611320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:38.612463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 633/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:38.617382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb18a9-608d-423d-afcf-0edbac2ef8e3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:38.621150 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:38.715729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb18a9-608d-423d-afcf-0edbac2ef8e3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:38.749097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd7746-3c84-46e6-9763-9a58c01cd1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:38.749692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd7746-3c84-46e6-9763-9a58c01cd1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:38.749966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd7746-3c84-46e6-9763-9a58c01cd1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:38.750266 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd7746-3c84-46e6-9763-9a58c01cd1e8 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:38.751088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd7746-3c84-46e6-9763-9a58c01cd1e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:38.751391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd7746-3c84-46e6-9763-9a58c01cd1e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:38.752804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd7746-3c84-46e6-9763-9a58c01cd1e8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:38.770161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 648/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 13 08:51:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:38.775767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919f44c-7268-44ac-88ae-735a30cd897a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:38.863546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:38.872789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919f44c-7268-44ac-88ae-735a30cd897a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:38.905870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b077e3-8c0c-451b-b38f-7a193a8229a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:38.906526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b077e3-8c0c-451b-b38f-7a193a8229a8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:38.906802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b077e3-8c0c-451b-b38f-7a193a8229a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:38.907072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b077e3-8c0c-451b-b38f-7a193a8229a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:38.971997 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b077e3-8c0c-451b-b38f-7a193a8229a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:38.973483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b077e3-8c0c-451b-b38f-7a193a8229a8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:39.058271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:39.068103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 634/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 13 08:51:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 08:51:39.074365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd7746-3c84-46e6-9763-9a58c01cd1e8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:39.137787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:39.174545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd7746-3c84-46e6-9763-9a58c01cd1e8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:39.207780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7972775-9718-45a4-8dde-3c818de9ef4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:39.208591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7972775-9718-45a4-8dde-3c818de9ef4d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:39.208867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7972775-9718-45a4-8dde-3c818de9ef4d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:39.209136 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7972775-9718-45a4-8dde-3c818de9ef4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:39.219143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7972775-9718-45a4-8dde-3c818de9ef4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:39.220704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7972775-9718-45a4-8dde-3c818de9ef4d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f7972775-9718-45a4-8dde-3c818de9ef4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 08:51:39.238702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:39.239774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:39.245494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 08:51:39.245494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 08:51:39.245494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 08:51:39.245494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:39.245494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 649/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 08:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:39.246385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b077e3-8c0c-451b-b38f-7a193a8229a8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:39.351022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b077e3-8c0c-451b-b38f-7a193a8229a8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:39.385631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8c2a9-bc84-449e-8f8d-6956babd8e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:39.386943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8c2a9-bc84-449e-8f8d-6956babd8e1c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:39.387236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8c2a9-bc84-449e-8f8d-6956babd8e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:39.387529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8c2a9-bc84-449e-8f8d-6956babd8e1c None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:39.388245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8c2a9-bc84-449e-8f8d-6956babd8e1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:39.388550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8c2a9-bc84-449e-8f8d-6956babd8e1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:39.389911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8c2a9-bc84-449e-8f8d-6956babd8e1c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:39.401136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 635/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 13 08:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:51:39.406953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7972775-9718-45a4-8dde-3c818de9ef4d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:39.505591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7972775-9718-45a4-8dde-3c818de9ef4d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:39.519869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:39.539540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8211f5-7e4e-48e2-9489-cfabaedb80fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:39.540316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8211f5-7e4e-48e2-9489-cfabaedb80fd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:39.540596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8211f5-7e4e-48e2-9489-cfabaedb80fd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:39.540869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8211f5-7e4e-48e2-9489-cfabaedb80fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:39.547893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8211f5-7e4e-48e2-9489-cfabaedb80fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:39.549318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8211f5-7e4e-48e2-9489-cfabaedb80fd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1a8211f5-7e4e-48e2-9489-cfabaedb80fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 08:51:39.563889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:39.564985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:39.565939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 08:51:39.565939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 08:51:39.565939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 08:51:39.565939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 08:51:39.565939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 08:51:39.565939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 08:51:39.565939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 08:51:39.565939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 13 08:51:39.565939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 650/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 13 08:51:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:51:39.571073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8c2a9-bc84-449e-8f8d-6956babd8e1c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:39.619562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:39.668682 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8c2a9-bc84-449e-8f8d-6956babd8e1c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:39.701947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a036bf0-7747-412c-9382-b4f367884e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:39.702647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a036bf0-7747-412c-9382-b4f367884e7e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:39.702926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a036bf0-7747-412c-9382-b4f367884e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:39.703196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a036bf0-7747-412c-9382-b4f367884e7e None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:39.703965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a036bf0-7747-412c-9382-b4f367884e7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:39.704270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a036bf0-7747-412c-9382-b4f367884e7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:39.705696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a036bf0-7747-412c-9382-b4f367884e7e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:39.718485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 636/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 13 08:51:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 08:51:39.724613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8211f5-7e4e-48e2-9489-cfabaedb80fd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:39.823847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8211f5-7e4e-48e2-9489-cfabaedb80fd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:39.857235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57673058-f4a0-4172-9437-75dbb2ba774f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:39.857902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57673058-f4a0-4172-9437-75dbb2ba774f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:39.858214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57673058-f4a0-4172-9437-75dbb2ba774f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:39.858490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57673058-f4a0-4172-9437-75dbb2ba774f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:39.859318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57673058-f4a0-4172-9437-75dbb2ba774f None admin] RBAC: Adding query filter params (role.id=2079875b2ff141a689d6eb49744f5c65, scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105, user.id=bca7c1164d3e4833a10ffc37cbc6f092) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:39.859624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57673058-f4a0-4172-9437-75dbb2ba774f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:39.861100 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57673058-f4a0-4172-9437-75dbb2ba774f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:39.866771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 651/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 13 08:51:39 2022] GET /identity/v3/role_assignments?user.id=bca7c1164d3e4833a10ffc37cbc6f092&scope.project.id=aa56bd5dbb81408c8fe362c5a1c02105&role.id=2079875b2ff141a689d6eb49744f5c65 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:41.926850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05329124-3f4a-4582-8e72-1a25804fede2 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:41.927429 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05329124-3f4a-4582-8e72-1a25804fede2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:41.927429 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05329124-3f4a-4582-8e72-1a25804fede2 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:41.928765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 637/1288] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:41.934378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad697a78-cee1-48c0-a395-1f365300047a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:41.934795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad697a78-cee1-48c0-a395-1f365300047a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:41.936889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad697a78-cee1-48c0-a395-1f365300047a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:41.953359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ad697a78-cee1-48c0-a395-1f365300047a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:41.954585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ad697a78-cee1-48c0-a395-1f365300047a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:41.983955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:42.020850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 652/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:42.043706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05329124-3f4a-4582-8e72-1a25804fede2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:42.142174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05329124-3f4a-4582-8e72-1a25804fede2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:42.152963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:42.176169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b2f46-52ce-42dd-8941-ee7d7da9eb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:42.176999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b2f46-52ce-42dd-8941-ee7d7da9eb78 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:42.177225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b2f46-52ce-42dd-8941-ee7d7da9eb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:42.177494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b2f46-52ce-42dd-8941-ee7d7da9eb78 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:42.178166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b2f46-52ce-42dd-8941-ee7d7da9eb78 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:42.179626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b2f46-52ce-42dd-8941-ee7d7da9eb78 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-646b2f46-52ce-42dd-8941-ee7d7da9eb78 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:42.191307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:42.192550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:42.193649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 638/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 13 08:51:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 08:51:42.198358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad697a78-cee1-48c0-a395-1f365300047a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:42.294668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad697a78-cee1-48c0-a395-1f365300047a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:42.326701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fa5d7-260d-4093-a519-ed783f325c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:42.327345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fa5d7-260d-4093-a519-ed783f325c90 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:42.327623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fa5d7-260d-4093-a519-ed783f325c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:42.327911 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fa5d7-260d-4093-a519-ed783f325c90 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:42.328587 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037fa5d7-260d-4093-a519-ed783f325c90 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:42.328886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037fa5d7-260d-4093-a519-ed783f325c90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:42.330263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037fa5d7-260d-4093-a519-ed783f325c90 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:42.334932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 653/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 13 08:51:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:42.341770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b2f46-52ce-42dd-8941-ee7d7da9eb78 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:42.440037 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b2f46-52ce-42dd-8941-ee7d7da9eb78 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:42.472376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd0971-d45e-4685-8dbb-996728656392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:42.473035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd0971-d45e-4685-8dbb-996728656392 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:42.473320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd0971-d45e-4685-8dbb-996728656392 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:42.473596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd0971-d45e-4685-8dbb-996728656392 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:42.474371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd0971-d45e-4685-8dbb-996728656392 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:42.480594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd0971-d45e-4685-8dbb-996728656392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:42.480594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd0971-d45e-4685-8dbb-996728656392 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:42.481198 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 639/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 13 08:51:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:44.545427 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01170b1-3e5a-47b1-a9e8-6213a7bbd466 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:44.545970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01170b1-3e5a-47b1-a9e8-6213a7bbd466 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:44.545970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01170b1-3e5a-47b1-a9e8-6213a7bbd466 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:44.547376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 654/1293] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:44.552813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663c8c5-d699-4cb8-b371-848c75da0a33 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:44.553204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663c8c5-d699-4cb8-b371-848c75da0a33 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:44.553530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663c8c5-d699-4cb8-b371-848c75da0a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:44.570775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8663c8c5-d699-4cb8-b371-848c75da0a33 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:44.571960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8663c8c5-d699-4cb8-b371-848c75da0a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:44.601968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:44.639221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 640/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:44.661848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01170b1-3e5a-47b1-a9e8-6213a7bbd466 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:44.693537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:44.761567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01170b1-3e5a-47b1-a9e8-6213a7bbd466 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:44.794599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fdf81-722b-48ee-a4e6-88b548451375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:44.795241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fdf81-722b-48ee-a4e6-88b548451375 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:44.795520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fdf81-722b-48ee-a4e6-88b548451375 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:44.795790 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fdf81-722b-48ee-a4e6-88b548451375 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:44.796504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fdf81-722b-48ee-a4e6-88b548451375 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:44.797846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fdf81-722b-48ee-a4e6-88b548451375 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:44.866806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 655/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 13 08:51:44 2022] POST /identity/v3/services => generated 246 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 08:51:46.904153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dc9db-c81a-462e-82f9-0dba968fb07a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:46.904689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dc9db-c81a-462e-82f9-0dba968fb07a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:46.904799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dc9db-c81a-462e-82f9-0dba968fb07a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:46.906294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 641/1296] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:46.911340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12db83c-13e3-494a-a526-74bddf1286eb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:46.911712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12db83c-13e3-494a-a526-74bddf1286eb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:46.911898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12db83c-13e3-494a-a526-74bddf1286eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:46.928587 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a12db83c-13e3-494a-a526-74bddf1286eb None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:46.929752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a12db83c-13e3-494a-a526-74bddf1286eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:47.001683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 656/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:47.024484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dc9db-c81a-462e-82f9-0dba968fb07a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:47.125774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dc9db-c81a-462e-82f9-0dba968fb07a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:47.159433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10700098-d324-4dd2-8f47-857154174ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:47.160160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10700098-d324-4dd2-8f47-857154174ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:47.160438 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10700098-d324-4dd2-8f47-857154174ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:47.160710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10700098-d324-4dd2-8f47-857154174ed0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:47.161369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10700098-d324-4dd2-8f47-857154174ed0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:47.163017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10700098-d324-4dd2-8f47-857154174ed0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-10700098-d324-4dd2-8f47-857154174ed0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:47.175269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:47.176323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:47.177187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 642/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 13 08:51:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:47.182591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12db83c-13e3-494a-a526-74bddf1286eb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:47.201337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:47.278393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12db83c-13e3-494a-a526-74bddf1286eb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:47.311443 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6482f-c459-4ab9-b50d-8dc520ce0868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:47.312086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6482f-c459-4ab9-b50d-8dc520ce0868 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:47.312362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6482f-c459-4ab9-b50d-8dc520ce0868 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:47.312633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6482f-c459-4ab9-b50d-8dc520ce0868 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:47.313326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6482f-c459-4ab9-b50d-8dc520ce0868 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:47.313627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6482f-c459-4ab9-b50d-8dc520ce0868 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:47.315011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6482f-c459-4ab9-b50d-8dc520ce0868 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:47.319965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 657/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 13 08:51:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:47.326380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10700098-d324-4dd2-8f47-857154174ed0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:47.358881 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:47.425552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10700098-d324-4dd2-8f47-857154174ed0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:47.457272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fc857-6c0d-4682-9669-4664f764f8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:47.457908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fc857-6c0d-4682-9669-4664f764f8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:47.458229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fc857-6c0d-4682-9669-4664f764f8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:47.458501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fc857-6c0d-4682-9669-4664f764f8b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:47.459305 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fc857-6c0d-4682-9669-4664f764f8b9 None admin] RBAC: Adding query filter params (interface=public, service_id=f5e15afb6b254d338766196e066bbe65, region_id=RegionOne) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:47.459820 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fc857-6c0d-4682-9669-4664f764f8b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:47.461499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fc857-6c0d-4682-9669-4664f764f8b9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:47.467695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 643/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 13 08:51:47 2022] GET /identity/v3/endpoints?service_id=f5e15afb6b254d338766196e066bbe65&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:47.473631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6482f-c459-4ab9-b50d-8dc520ce0868 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:47.555488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:47.572894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6482f-c459-4ab9-b50d-8dc520ce0868 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:47.606404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3a704-8bfe-4686-ae99-650bdb0064fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:47.607050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3a704-8bfe-4686-ae99-650bdb0064fc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:47.607327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3a704-8bfe-4686-ae99-650bdb0064fc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:47.607641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3a704-8bfe-4686-ae99-650bdb0064fc None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:47.608306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3a704-8bfe-4686-ae99-650bdb0064fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:47.609658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3a704-8bfe-4686-ae99-650bdb0064fc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:47.616301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 658/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 13 08:51:47 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 08:51:49.630850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5fbf7-bd6f-45f3-830e-9f15b3293be5 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:49.631402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5fbf7-bd6f-45f3-830e-9f15b3293be5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:49.631510 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5fbf7-bd6f-45f3-830e-9f15b3293be5 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:49.632777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 644/1302] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:51:49.638146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58887100-6f74-4146-8411-1d6a8fbd3f88 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:49.638521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58887100-6f74-4146-8411-1d6a8fbd3f88 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:49.638696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58887100-6f74-4146-8411-1d6a8fbd3f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:49.655068 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-58887100-6f74-4146-8411-1d6a8fbd3f88 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:51:49.656213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-58887100-6f74-4146-8411-1d6a8fbd3f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:51:49.721130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 659/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:51:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 08:51:49.744391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5fbf7-bd6f-45f3-830e-9f15b3293be5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:49.838328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:49.845374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5fbf7-bd6f-45f3-830e-9f15b3293be5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:49.879117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccbd8d-893e-4b5f-8504-bab3198ace2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:49.879905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccbd8d-893e-4b5f-8504-bab3198ace2c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:49.880182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccbd8d-893e-4b5f-8504-bab3198ace2c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:49.880448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccbd8d-893e-4b5f-8504-bab3198ace2c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:49.881138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccbd8d-893e-4b5f-8504-bab3198ace2c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:49.882485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccbd8d-893e-4b5f-8504-bab3198ace2c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-72ccbd8d-893e-4b5f-8504-bab3198ace2c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 08:51:49.894343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 08:51:49.895372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 13 08:51:49.896496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 645/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 13 08:51:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 08:51:49.902193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58887100-6f74-4146-8411-1d6a8fbd3f88 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:49.977929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:51:50.001044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58887100-6f74-4146-8411-1d6a8fbd3f88 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:50.035813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59046810-c8fd-4209-8a01-19503023f568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:50.036473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59046810-c8fd-4209-8a01-19503023f568 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:50.036757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59046810-c8fd-4209-8a01-19503023f568 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:50.037033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59046810-c8fd-4209-8a01-19503023f568 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:50.038003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59046810-c8fd-4209-8a01-19503023f568 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 08:51:50.038342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59046810-c8fd-4209-8a01-19503023f568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:50.039709 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59046810-c8fd-4209-8a01-19503023f568 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:50.044943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 660/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 13 08:51:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 08:51:50.050705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccbd8d-893e-4b5f-8504-bab3198ace2c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:51:50.147992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccbd8d-893e-4b5f-8504-bab3198ace2c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:51:50.180486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77d684-4814-45e9-9a1e-04ee90cf7737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:51:50.181122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77d684-4814-45e9-9a1e-04ee90cf7737 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:51:50.181400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77d684-4814-45e9-9a1e-04ee90cf7737 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:51:50.181669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77d684-4814-45e9-9a1e-04ee90cf7737 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:51:50.182428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77d684-4814-45e9-9a1e-04ee90cf7737 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:51:50.183786 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77d684-4814-45e9-9a1e-04ee90cf7737 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:51:50.249560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 646/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 13 08:51:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 08:52:00.041762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=30 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:52:00.244576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:23.619318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635ff9b-e84c-4466-930d-a700d9b3cc93 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:23.620385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635ff9b-e84c-4466-930d-a700d9b3cc93 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:23.620900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635ff9b-e84c-4466-930d-a700d9b3cc93 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:23.628645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c8143-8253-4509-9dc0-c170eb010e3a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:23.630932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c8143-8253-4509-9dc0-c170eb010e3a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:23.630932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c8143-8253-4509-9dc0-c170eb010e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:23.633690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 661/1307] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:23.725840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6f6c8143-8253-4509-9dc0-c170eb010e3a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:23.727481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6f6c8143-8253-4509-9dc0-c170eb010e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:23.862765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 647/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:23.900255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec9fae-d784-4ed5-bbd6-d8d2b698aad8 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:23.900891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec9fae-d784-4ed5-bbd6-d8d2b698aad8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:23.901392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec9fae-d784-4ed5-bbd6-d8d2b698aad8 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:23.903797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 662/1309] 10.222.0.40 () {54 vars in 964 bytes} [Wed Jul 13 08:57:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:23.910075 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bcdd4-c06f-4f6b-8292-d115c8f09374 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:23.910362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bcdd4-c06f-4f6b-8292-d115c8f09374 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:23.910642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bcdd4-c06f-4f6b-8292-d115c8f09374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:23.995130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-651bcdd4-c06f-4f6b-8292-d115c8f09374 None None] MFA Rules not processed for user `12824152139841138f457dfe5c9758e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:23.996667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-651bcdd4-c06f-4f6b-8292-d115c8f09374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:24.021254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:24.084521 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 648/1310] 10.222.0.40 () {58 vars in 1061 bytes} [Wed Jul 13 08:57:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:24.094162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec9fae-d784-4ed5-bbd6-d8d2b698aad8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:24.095900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-70ec9fae-d784-4ed5-bbd6-d8d2b698aad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:24.225708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec9fae-d784-4ed5-bbd6-d8d2b698aad8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:24.310702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d4583-f24c-4fe2-8c8b-a72bcfb28448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:24.311358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d4583-f24c-4fe2-8c8b-a72bcfb28448 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:24.311640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d4583-f24c-4fe2-8c8b-a72bcfb28448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:24.311915 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d4583-f24c-4fe2-8c8b-a72bcfb28448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:24.313585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-981d4583-f24c-4fe2-8c8b-a72bcfb28448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:24.382832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:24.440150 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d4583-f24c-4fe2-8c8b-a72bcfb28448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:24.441584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d4583-f24c-4fe2-8c8b-a72bcfb28448 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:24.464000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 663/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:57:26.799119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b24ce5-bb1b-4b1c-81c1-d12eaf42b554 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:26.799673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b24ce5-bb1b-4b1c-81c1-d12eaf42b554 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:26.799673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b24ce5-bb1b-4b1c-81c1-d12eaf42b554 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:26.802392 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 649/1312] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:26.806302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb0c46-582c-4489-9d69-2a741897e815 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:26.807165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb0c46-582c-4489-9d69-2a741897e815 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:26.807638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb0c46-582c-4489-9d69-2a741897e815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:26.892063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-89eb0c46-582c-4489-9d69-2a741897e815 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:26.893412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-89eb0c46-582c-4489-9d69-2a741897e815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:27.005077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 664/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:27.044851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b24ce5-bb1b-4b1c-81c1-d12eaf42b554 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:27.046436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f7b24ce5-bb1b-4b1c-81c1-d12eaf42b554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:27.164779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b24ce5-bb1b-4b1c-81c1-d12eaf42b554 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:27.241270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:27.244206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d3aaf-bbdb-44c9-bc4d-fcf29d57269f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:27.244852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d3aaf-bbdb-44c9-bc4d-fcf29d57269f service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:27.245134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d3aaf-bbdb-44c9-bc4d-fcf29d57269f service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:27.245396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d3aaf-bbdb-44c9-bc4d-fcf29d57269f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:27.247115 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-db3d3aaf-bbdb-44c9-bc4d-fcf29d57269f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:27.373089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d3aaf-bbdb-44c9-bc4d-fcf29d57269f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:27.374653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d3aaf-bbdb-44c9-bc4d-fcf29d57269f service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:27.394941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 650/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:57:29.859262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb5f83-a754-4f78-b9a5-e513c353183c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:29.859806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb5f83-a754-4f78-b9a5-e513c353183c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:29.859806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb5f83-a754-4f78-b9a5-e513c353183c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:29.861164 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 665/1315] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:29.866135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31e046-f841-492d-8dfa-719cc20755a2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:29.866519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31e046-f841-492d-8dfa-719cc20755a2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:29.866680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31e046-f841-492d-8dfa-719cc20755a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:29.930761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fd31e046-f841-492d-8dfa-719cc20755a2 None None] MFA Rules not processed for user `654695f326044efdb73fe2956b7ce735`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:29.931969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fd31e046-f841-492d-8dfa-719cc20755a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:29.984254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:30.028563 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 651/1316] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:30.067269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb5f83-a754-4f78-b9a5-e513c353183c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:30.068786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ebcb5f83-a754-4f78-b9a5-e513c353183c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:30.079208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:30.171180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb5f83-a754-4f78-b9a5-e513c353183c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:30.243111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2c386-18c2-42f9-b943-baea8d90e67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:30.243780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2c386-18c2-42f9-b943-baea8d90e67b service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:30.244062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2c386-18c2-42f9-b943-baea8d90e67b service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:30.244339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2c386-18c2-42f9-b943-baea8d90e67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:30.245964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3ef2c386-18c2-42f9-b943-baea8d90e67b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:30.307240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:30.354455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2c386-18c2-42f9-b943-baea8d90e67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:30.356429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2c386-18c2-42f9-b943-baea8d90e67b service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:30.375445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 666/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:57:33.112381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bf363-78e5-4c2e-a6cb-ce2cce14e23a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:33.112918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bf363-78e5-4c2e-a6cb-ce2cce14e23a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:33.112918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bf363-78e5-4c2e-a6cb-ce2cce14e23a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:33.114322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 652/1318] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:33.119487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187bbf7-03ff-4b30-8ede-34beeba87b12 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:33.119912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187bbf7-03ff-4b30-8ede-34beeba87b12 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:33.120043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187bbf7-03ff-4b30-8ede-34beeba87b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:33.185362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9187bbf7-03ff-4b30-8ede-34beeba87b12 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:33.186578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9187bbf7-03ff-4b30-8ede-34beeba87b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:33.281576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 667/1319] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:33.319582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bf363-78e5-4c2e-a6cb-ce2cce14e23a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:33.321106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a18bf363-78e5-4c2e-a6cb-ce2cce14e23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:33.424942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bf363-78e5-4c2e-a6cb-ce2cce14e23a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:33.495940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7302d82-c299-4b64-b2ec-8257b680e4f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:33.497421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7302d82-c299-4b64-b2ec-8257b680e4f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:33.497704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7302d82-c299-4b64-b2ec-8257b680e4f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:33.497979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7302d82-c299-4b64-b2ec-8257b680e4f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:33.499641 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d7302d82-c299-4b64-b2ec-8257b680e4f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:33.506970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:33.608407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7302d82-c299-4b64-b2ec-8257b680e4f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:33.609833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7302d82-c299-4b64-b2ec-8257b680e4f1 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:33.628521 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 653/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:57:36.995179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1702049-1eca-42c8-9b1a-f00663b60cca None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:36.995796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1702049-1eca-42c8-9b1a-f00663b60cca None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:36.995796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1702049-1eca-42c8-9b1a-f00663b60cca None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:36.997797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 668/1321] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:37.003095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b4b0b-83ca-4c32-a2ae-bf9c852a148b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:37.003555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b4b0b-83ca-4c32-a2ae-bf9c852a148b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:37.003640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b4b0b-83ca-4c32-a2ae-bf9c852a148b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:37.074410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a33b4b0b-83ca-4c32-a2ae-bf9c852a148b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:37.076332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a33b4b0b-83ca-4c32-a2ae-bf9c852a148b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:37.139336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:37.182360 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 654/1322] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:37.219889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1702049-1eca-42c8-9b1a-f00663b60cca None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:37.221412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e1702049-1eca-42c8-9b1a-f00663b60cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:37.242074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:37.328232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1702049-1eca-42c8-9b1a-f00663b60cca None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:37.405121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d5654-c9f6-4e7b-890b-295bae6b87a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:37.405764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d5654-c9f6-4e7b-890b-295bae6b87a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:37.406127 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d5654-c9f6-4e7b-890b-295bae6b87a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:37.406344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d5654-c9f6-4e7b-890b-295bae6b87a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:37.408007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a10d5654-c9f6-4e7b-890b-295bae6b87a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:37.481281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:37.518289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d5654-c9f6-4e7b-890b-295bae6b87a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:37.519725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d5654-c9f6-4e7b-890b-295bae6b87a0 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:37.538426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 669/1323] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:57:39.630817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6142961-6b57-4d95-8334-ae1fe7acd9e0 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:39.631380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6142961-6b57-4d95-8334-ae1fe7acd9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:39.631380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6142961-6b57-4d95-8334-ae1fe7acd9e0 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:39.632718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 655/1324] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:39.637799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da49e78-0c14-481c-97a3-95893a053bfb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:39.638288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da49e78-0c14-481c-97a3-95893a053bfb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:39.638396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da49e78-0c14-481c-97a3-95893a053bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:39.704019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9da49e78-0c14-481c-97a3-95893a053bfb None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:39.705208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9da49e78-0c14-481c-97a3-95893a053bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:39.804550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 670/1325] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:39.854784 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27618ff4-9e6c-483c-ac53-25dedbbdf705 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:39.855063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27618ff4-9e6c-483c-ac53-25dedbbdf705 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:39.855383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27618ff4-9e6c-483c-ac53-25dedbbdf705 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:39.858058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 656/1326] 10.222.0.40 () {54 vars in 964 bytes} [Wed Jul 13 08:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:39.864388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca42ea-53ba-428d-be1f-db7e30934e8e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:39.864737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca42ea-53ba-428d-be1f-db7e30934e8e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:39.873161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca42ea-53ba-428d-be1f-db7e30934e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:39.895835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:39.942758 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-48ca42ea-53ba-428d-be1f-db7e30934e8e None None] MFA Rules not processed for user `12824152139841138f457dfe5c9758e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:39.943974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-48ca42ea-53ba-428d-be1f-db7e30934e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:40.027277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 671/1327] 10.222.0.40 () {58 vars in 1061 bytes} [Wed Jul 13 08:57:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:40.037094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27618ff4-9e6c-483c-ac53-25dedbbdf705 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:40.038746 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-27618ff4-9e6c-483c-ac53-25dedbbdf705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:40.151403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27618ff4-9e6c-483c-ac53-25dedbbdf705 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:40.230534 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d9993-dc7d-4c85-a1db-d78bd427e238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:40.231650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d9993-dc7d-4c85-a1db-d78bd427e238 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:40.231932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d9993-dc7d-4c85-a1db-d78bd427e238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:40.232207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d9993-dc7d-4c85-a1db-d78bd427e238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:40.233843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-836d9993-dc7d-4c85-a1db-d78bd427e238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:40.252250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:40.349778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d9993-dc7d-4c85-a1db-d78bd427e238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:40.351263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d9993-dc7d-4c85-a1db-d78bd427e238 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:40.369815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 657/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:57:42.702522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b52df-6909-40ab-940e-f01738051dfa None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:42.703038 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b52df-6909-40ab-940e-f01738051dfa None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:42.703131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b52df-6909-40ab-940e-f01738051dfa None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:42.704470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 672/1329] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:42.709731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b81235-e674-4659-877a-0cb05a256144 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:42.710173 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b81235-e674-4659-877a-0cb05a256144 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:42.710370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b81235-e674-4659-877a-0cb05a256144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:42.794966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-90b81235-e674-4659-877a-0cb05a256144 None None] MFA Rules not processed for user `654695f326044efdb73fe2956b7ce735`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:42.796289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-90b81235-e674-4659-877a-0cb05a256144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:42.865233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:42.896541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 658/1330] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:42.934157 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b52df-6909-40ab-940e-f01738051dfa None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:42.935581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9f0b52df-6909-40ab-940e-f01738051dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:43.039467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b52df-6909-40ab-940e-f01738051dfa None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:43.059079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:43.113813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d4db5-a294-426b-b30b-240bb41dc85a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:43.114490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d4db5-a294-426b-b30b-240bb41dc85a service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:43.114773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d4db5-a294-426b-b30b-240bb41dc85a service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:43.115047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d4db5-a294-426b-b30b-240bb41dc85a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:43.116678 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7d3d4db5-a294-426b-b30b-240bb41dc85a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:43.224368 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d4db5-a294-426b-b30b-240bb41dc85a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:43.225801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d4db5-a294-426b-b30b-240bb41dc85a service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:43.243818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 673/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:57:46.023764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b67d1-02ab-4e17-9a93-dcc5cb82c20e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:46.024316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b67d1-02ab-4e17-9a93-dcc5cb82c20e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:46.024413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b67d1-02ab-4e17-9a93-dcc5cb82c20e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:46.025703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 659/1332] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:46.031221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb11818-4fe7-4e40-8867-b9392d3a0698 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:46.031634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb11818-4fe7-4e40-8867-b9392d3a0698 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:46.031784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb11818-4fe7-4e40-8867-b9392d3a0698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:46.094715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:46.097386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2bb11818-4fe7-4e40-8867-b9392d3a0698 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:46.098609 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2bb11818-4fe7-4e40-8867-b9392d3a0698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:46.196016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 674/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:46.233116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b67d1-02ab-4e17-9a93-dcc5cb82c20e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:46.234698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-822b67d1-02ab-4e17-9a93-dcc5cb82c20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:46.336846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b67d1-02ab-4e17-9a93-dcc5cb82c20e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:46.421602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589c63e-afe6-4aed-b654-343a1d99351b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:46.421602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589c63e-afe6-4aed-b654-343a1d99351b service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:46.421602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589c63e-afe6-4aed-b654-343a1d99351b service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:46.421602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589c63e-afe6-4aed-b654-343a1d99351b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:46.422109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4589c63e-afe6-4aed-b654-343a1d99351b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:46.445723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:46.526254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589c63e-afe6-4aed-b654-343a1d99351b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:46.528225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589c63e-afe6-4aed-b654-343a1d99351b service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:46.546489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 660/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:57:48.620729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebeae64-0783-4f9c-a0c3-dbdbde1d582c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:48.621919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebeae64-0783-4f9c-a0c3-dbdbde1d582c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:48.622049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebeae64-0783-4f9c-a0c3-dbdbde1d582c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:48.623519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 675/1335] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:48.629111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b0c27-4179-4ab3-8e5d-ea2bec149bcc None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:48.629564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b0c27-4179-4ab3-8e5d-ea2bec149bcc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:48.629690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b0c27-4179-4ab3-8e5d-ea2bec149bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:48.693497 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a46b0c27-4179-4ab3-8e5d-ea2bec149bcc None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:48.694711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a46b0c27-4179-4ab3-8e5d-ea2bec149bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:48.768230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:48.790715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 661/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:48.831402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebeae64-0783-4f9c-a0c3-dbdbde1d582c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:48.832928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9ebeae64-0783-4f9c-a0c3-dbdbde1d582c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:48.938359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebeae64-0783-4f9c-a0c3-dbdbde1d582c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:48.966023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:49.010524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce1ada-e964-49fa-94cd-363ca41746b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:49.011158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce1ada-e964-49fa-94cd-363ca41746b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:49.011574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce1ada-e964-49fa-94cd-363ca41746b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:49.012103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce1ada-e964-49fa-94cd-363ca41746b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:49.013750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-33ce1ada-e964-49fa-94cd-363ca41746b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:49.120684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce1ada-e964-49fa-94cd-363ca41746b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:49.122734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce1ada-e964-49fa-94cd-363ca41746b0 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:49.140902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 676/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:57:51.900976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4f6b6-17fd-40d9-8258-46e6e671dd67 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:51.901543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4f6b6-17fd-40d9-8258-46e6e671dd67 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:51.901543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4f6b6-17fd-40d9-8258-46e6e671dd67 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:51.903046 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 662/1338] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:51.908395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0f97d-f3c0-4b4b-94be-cd565c778a8a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:51.909096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0f97d-f3c0-4b4b-94be-cd565c778a8a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:51.909198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0f97d-f3c0-4b4b-94be-cd565c778a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:51.973026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-70e0f97d-f3c0-4b4b-94be-cd565c778a8a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:51.974241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-70e0f97d-f3c0-4b4b-94be-cd565c778a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:51.983434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:52.072403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 677/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:52.108715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4f6b6-17fd-40d9-8258-46e6e671dd67 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:52.110259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0de4f6b6-17fd-40d9-8258-46e6e671dd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:52.212585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4f6b6-17fd-40d9-8258-46e6e671dd67 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:52.284583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d0fba-0fee-4987-9a7d-a5e5e903251f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:52.285221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d0fba-0fee-4987-9a7d-a5e5e903251f service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:52.285501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d0fba-0fee-4987-9a7d-a5e5e903251f service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:52.285776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d0fba-0fee-4987-9a7d-a5e5e903251f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:52.287549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a10d0fba-0fee-4987-9a7d-a5e5e903251f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:52.323551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:52.396710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d0fba-0fee-4987-9a7d-a5e5e903251f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:52.398183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d0fba-0fee-4987-9a7d-a5e5e903251f service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:52.415757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 663/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:57:56.720736 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59c3b3-9ef8-40f0-bd45-a3cf8d98f974 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:56.721159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59c3b3-9ef8-40f0-bd45-a3cf8d98f974 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:56.721285 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59c3b3-9ef8-40f0-bd45-a3cf8d98f974 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:56.722670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 678/1341] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:57:56.728126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f151c-c3c5-4553-978b-626716f5aca0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:56.728551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f151c-c3c5-4553-978b-626716f5aca0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:56.728715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f151c-c3c5-4553-978b-626716f5aca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:56.813694 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-aa6f151c-c3c5-4553-978b-626716f5aca0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:57:56.815168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-aa6f151c-c3c5-4553-978b-626716f5aca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:56.903968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:56.915232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 664/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:57:56.955801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59c3b3-9ef8-40f0-bd45-a3cf8d98f974 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:57:56.957358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7e59c3b3-9ef8-40f0-bd45-a3cf8d98f974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:57.063229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59c3b3-9ef8-40f0-bd45-a3cf8d98f974 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:57:57.100315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:57:57.137376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afcfaf-fdfa-4408-9586-c1de91bfe025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:57:57.138020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afcfaf-fdfa-4408-9586-c1de91bfe025 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:57:57.138327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afcfaf-fdfa-4408-9586-c1de91bfe025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:57:57.138599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afcfaf-fdfa-4408-9586-c1de91bfe025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:57:57.140245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-95afcfaf-fdfa-4408-9586-c1de91bfe025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:57:57.251734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95afcfaf-fdfa-4408-9586-c1de91bfe025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:57:57.253162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95afcfaf-fdfa-4408-9586-c1de91bfe025 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:57:57.271887 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 679/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:57:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:00.283028 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfceee5-eef6-4e5a-9d05-e5cccaf21474 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:00.283593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfceee5-eef6-4e5a-9d05-e5cccaf21474 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:00.283593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfceee5-eef6-4e5a-9d05-e5cccaf21474 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:00.285352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 665/1344] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:00.290396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24facf-4014-4924-a1ca-306818e2b94b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:00.290794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24facf-4014-4924-a1ca-306818e2b94b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:00.291102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24facf-4014-4924-a1ca-306818e2b94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:00.357856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7e24facf-4014-4924-a1ca-306818e2b94b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:00.359106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7e24facf-4014-4924-a1ca-306818e2b94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:00.377915 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:00.457256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 680/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:00.495338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfceee5-eef6-4e5a-9d05-e5cccaf21474 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:00.497136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fbfceee5-eef6-4e5a-9d05-e5cccaf21474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:00.600280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfceee5-eef6-4e5a-9d05-e5cccaf21474 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:00.675898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b32e87-fe3d-4e24-acd8-c2514c7d3345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:00.676547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b32e87-fe3d-4e24-acd8-c2514c7d3345 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:00.676828 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b32e87-fe3d-4e24-acd8-c2514c7d3345 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:00.677100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b32e87-fe3d-4e24-acd8-c2514c7d3345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:00.678769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-21b32e87-fe3d-4e24-acd8-c2514c7d3345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:00.725471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:00.797368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b32e87-fe3d-4e24-acd8-c2514c7d3345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:00.798977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b32e87-fe3d-4e24-acd8-c2514c7d3345 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:00.818893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 666/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:04.955693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081044bc-8947-4642-8eb2-fefb2bb55ecc None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:04.956199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081044bc-8947-4642-8eb2-fefb2bb55ecc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:04.956290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081044bc-8947-4642-8eb2-fefb2bb55ecc None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:04.957620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 681/1347] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:04.963065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff07017-e73b-4e1a-b0ea-4405de5c4793 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:04.963506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff07017-e73b-4e1a-b0ea-4405de5c4793 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:04.963617 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff07017-e73b-4e1a-b0ea-4405de5c4793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:05.032955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-aff07017-e73b-4e1a-b0ea-4405de5c4793 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:05.034380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-aff07017-e73b-4e1a-b0ea-4405de5c4793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:05.136658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:05.139486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 667/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:05.184218 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081044bc-8947-4642-8eb2-fefb2bb55ecc None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:05.185770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-081044bc-8947-4642-8eb2-fefb2bb55ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:05.294829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081044bc-8947-4642-8eb2-fefb2bb55ecc None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:05.344287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:05.372721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0866c-ca34-4657-984f-72882a60c28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:05.373639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0866c-ca34-4657-984f-72882a60c28d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:05.373930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0866c-ca34-4657-984f-72882a60c28d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:05.374300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0866c-ca34-4657-984f-72882a60c28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:05.376317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-89b0866c-ca34-4657-984f-72882a60c28d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:05.501027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0866c-ca34-4657-984f-72882a60c28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:05.502574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0866c-ca34-4657-984f-72882a60c28d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:05.522130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 682/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:10.569131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09419c6-42e8-483d-a595-4af986c4f2fc None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:10.570211 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09419c6-42e8-483d-a595-4af986c4f2fc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:10.570718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09419c6-42e8-483d-a595-4af986c4f2fc None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:10.572651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 668/1350] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 08:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:10.579201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea09669-bb33-4277-bd36-2a3f0c294b66 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:10.580087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea09669-bb33-4277-bd36-2a3f0c294b66 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:10.580604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea09669-bb33-4277-bd36-2a3f0c294b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:10.706757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bea09669-bb33-4277-bd36-2a3f0c294b66 None None] MFA Rules not processed for user `bca7c1164d3e4833a10ffc37cbc6f092`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:10.707945 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bea09669-bb33-4277-bd36-2a3f0c294b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:10.736097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:10.843874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 683/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 08:58:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:14.959238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7f042-5aac-4d1f-a439-73a0d8f1c64c None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:14.959775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7f042-5aac-4d1f-a439-73a0d8f1c64c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:14.959775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7f042-5aac-4d1f-a439-73a0d8f1c64c None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:14.961292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 669/1352] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 08:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:14.966572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef767c5-b05f-4031-b68f-6ee8a0a85d52 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:14.967404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef767c5-b05f-4031-b68f-6ee8a0a85d52 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:14.967913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef767c5-b05f-4031-b68f-6ee8a0a85d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:15.052211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8ef767c5-b05f-4031-b68f-6ee8a0a85d52 None None] MFA Rules not processed for user `bca7c1164d3e4833a10ffc37cbc6f092`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:15.054458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8ef767c5-b05f-4031-b68f-6ee8a0a85d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:15.147737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:15.192097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 684/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 08:58:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:16.411742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbbea3-2752-4ec4-8197-b977c38e6196 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:16.412303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbbea3-2752-4ec4-8197-b977c38e6196 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:16.412303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbbea3-2752-4ec4-8197-b977c38e6196 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:16.413552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 670/1354] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 08:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:16.418617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad159e-9148-4874-b223-262318604155 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:16.419042 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad159e-9148-4874-b223-262318604155 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:16.419202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad159e-9148-4874-b223-262318604155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:16.495723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-34ad159e-9148-4874-b223-262318604155 None None] MFA Rules not processed for user `bca7c1164d3e4833a10ffc37cbc6f092`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:16.496936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-34ad159e-9148-4874-b223-262318604155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:16.610770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 685/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 08:58:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:20.201331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b479e7-55aa-4483-8ae9-db6773eec031 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:20.203748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b479e7-55aa-4483-8ae9-db6773eec031 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:20.203748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b479e7-55aa-4483-8ae9-db6773eec031 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:20.203952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 671/1356] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:20.209671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a01ee-f1bb-4355-a39a-15757f0579c9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:20.210526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a01ee-f1bb-4355-a39a-15757f0579c9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:20.211034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a01ee-f1bb-4355-a39a-15757f0579c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:20.252014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:20.288164 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-720a01ee-f1bb-4355-a39a-15757f0579c9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:20.289361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-720a01ee-f1bb-4355-a39a-15757f0579c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:20.400320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 686/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:20.456662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce878a36-277c-4c28-99e3-b714c9aed8a8 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:20.457302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce878a36-277c-4c28-99e3-b714c9aed8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:20.457877 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce878a36-277c-4c28-99e3-b714c9aed8a8 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:20.459688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 672/1358] 10.222.0.40 () {54 vars in 961 bytes} [Wed Jul 13 08:58:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:20.476172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619412d1-ad63-4242-8d18-85c05cb9c378 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:20.476479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619412d1-ad63-4242-8d18-85c05cb9c378 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:20.476907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619412d1-ad63-4242-8d18-85c05cb9c378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:20.574971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-619412d1-ad63-4242-8d18-85c05cb9c378 None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:20.576306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-619412d1-ad63-4242-8d18-85c05cb9c378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:20.631787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:20.706112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 687/1359] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Jul 13 08:58:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:20.714443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce878a36-277c-4c28-99e3-b714c9aed8a8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:20.715915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ce878a36-277c-4c28-99e3-b714c9aed8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:20.878894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce878a36-277c-4c28-99e3-b714c9aed8a8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:20.961181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:21.003809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fb6e0-2560-4485-b9fc-9c1a76e5236b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:21.004452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fb6e0-2560-4485-b9fc-9c1a76e5236b service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:21.004731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fb6e0-2560-4485-b9fc-9c1a76e5236b service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:21.004999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fb6e0-2560-4485-b9fc-9c1a76e5236b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:21.006649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d93fb6e0-2560-4485-b9fc-9c1a76e5236b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:21.105218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93fb6e0-2560-4485-b9fc-9c1a76e5236b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:21.106839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93fb6e0-2560-4485-b9fc-9c1a76e5236b service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:21.127148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 673/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:23.387285 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c414f-0ca8-4d7e-884d-55307f6380b1 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:23.387787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c414f-0ca8-4d7e-884d-55307f6380b1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:23.387874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c414f-0ca8-4d7e-884d-55307f6380b1 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:23.389418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 688/1361] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:23.394511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382ed1f-c6d2-41b4-9d43-829b06fc6ca1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:23.394944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382ed1f-c6d2-41b4-9d43-829b06fc6ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:23.395065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382ed1f-c6d2-41b4-9d43-829b06fc6ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:23.468620 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3382ed1f-c6d2-41b4-9d43-829b06fc6ca1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:23.469983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3382ed1f-c6d2-41b4-9d43-829b06fc6ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:23.509230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:23.576350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 674/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:23.622169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c414f-0ca8-4d7e-884d-55307f6380b1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:23.623767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b09c414f-0ca8-4d7e-884d-55307f6380b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:23.801295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c414f-0ca8-4d7e-884d-55307f6380b1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:23.811894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:23.937894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991717d-c43f-4d49-969e-18ff625e8331 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:23.938567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991717d-c43f-4d49-969e-18ff625e8331 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:23.938893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991717d-c43f-4d49-969e-18ff625e8331 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:23.939106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991717d-c43f-4d49-969e-18ff625e8331 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:23.940716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9991717d-c43f-4d49-969e-18ff625e8331 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:24.046198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991717d-c43f-4d49-969e-18ff625e8331 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:24.047938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991717d-c43f-4d49-969e-18ff625e8331 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:24.067901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 689/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:26.576424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a403d2-a06b-4970-b44e-b1807cbac005 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:26.577285 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a403d2-a06b-4970-b44e-b1807cbac005 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:26.577566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a403d2-a06b-4970-b44e-b1807cbac005 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:26.578963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 675/1364] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:26.584848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c611a7-ff0e-4a25-9028-476390fded1c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:26.585282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c611a7-ff0e-4a25-9028-476390fded1c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:26.589210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c611a7-ff0e-4a25-9028-476390fded1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:26.602868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:26.664475 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-43c611a7-ff0e-4a25-9028-476390fded1c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:26.665729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-43c611a7-ff0e-4a25-9028-476390fded1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:26.775285 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 690/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:26.823450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4073fe1-69a5-4400-ab83-8f481bcfc977 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:26.823747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4073fe1-69a5-4400-ab83-8f481bcfc977 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:26.824022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4073fe1-69a5-4400-ab83-8f481bcfc977 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:26.829993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 676/1366] 10.222.0.40 () {54 vars in 961 bytes} [Wed Jul 13 08:58:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:26.833990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba53ec4-b004-4b9a-9a6f-ae04cec4f0f1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:26.834399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba53ec4-b004-4b9a-9a6f-ae04cec4f0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:26.834809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba53ec4-b004-4b9a-9a6f-ae04cec4f0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:26.919542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1ba53ec4-b004-4b9a-9a6f-ae04cec4f0f1 None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:26.921034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1ba53ec4-b004-4b9a-9a6f-ae04cec4f0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:26.958785 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:27.053823 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 691/1367] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Jul 13 08:58:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:27.061122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4073fe1-69a5-4400-ab83-8f481bcfc977 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:27.062804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b4073fe1-69a5-4400-ab83-8f481bcfc977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:27.226246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4073fe1-69a5-4400-ab83-8f481bcfc977 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:27.249930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:27.355862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4c961-2074-491f-b9d3-92230e129d2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:27.356512 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4c961-2074-491f-b9d3-92230e129d2b service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:27.356792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4c961-2074-491f-b9d3-92230e129d2b service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:27.357064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4c961-2074-491f-b9d3-92230e129d2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:27.358790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c2e4c961-2074-491f-b9d3-92230e129d2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:27.458938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4c961-2074-491f-b9d3-92230e129d2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:27.460555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4c961-2074-491f-b9d3-92230e129d2b service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:27.481625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 677/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:28.424152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17371d-beb2-4385-9e9f-c05d5232a2ee None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:28.424673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17371d-beb2-4385-9e9f-c05d5232a2ee None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:28.424780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17371d-beb2-4385-9e9f-c05d5232a2ee None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:28.426276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 692/1369] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:28.431295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af33537-a40d-43a1-aa75-043379734ab6 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:28.431706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af33537-a40d-43a1-aa75-043379734ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:28.431918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af33537-a40d-43a1-aa75-043379734ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:28.463768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:28.501245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3af33537-a40d-43a1-aa75-043379734ab6 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:28.502634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3af33537-a40d-43a1-aa75-043379734ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:28.734081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 678/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:29.072252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea479db-30ef-4671-8b5c-7f580a94f4c4 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:29.074008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea479db-30ef-4671-8b5c-7f580a94f4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:29.074008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea479db-30ef-4671-8b5c-7f580a94f4c4 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:29.074949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 693/1371] 10.222.0.40 () {54 vars in 963 bytes} [Wed Jul 13 08:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:29.079749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098fe80-238d-4979-b732-ddc5afdd64eb None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:29.080046 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098fe80-238d-4979-b732-ddc5afdd64eb None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:29.080320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098fe80-238d-4979-b732-ddc5afdd64eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:29.157257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8098fe80-238d-4979-b732-ddc5afdd64eb None None] MFA Rules not processed for user `b3f12750f91645748b37e2b159a9bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:29.158728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8098fe80-238d-4979-b732-ddc5afdd64eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:29.210668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:29.241156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 679/1372] 10.222.0.40 () {58 vars in 1060 bytes} [Wed Jul 13 08:58:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:29.249374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea479db-30ef-4671-8b5c-7f580a94f4c4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:29.250902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cea479db-30ef-4671-8b5c-7f580a94f4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:29.366921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea479db-30ef-4671-8b5c-7f580a94f4c4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:29.412507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:29.451265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dc511-3e8b-488b-8a92-f71478fc413a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:29.451750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dc511-3e8b-488b-8a92-f71478fc413a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:29.451985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dc511-3e8b-488b-8a92-f71478fc413a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:29.452222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dc511-3e8b-488b-8a92-f71478fc413a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:29.458631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-874dc511-3e8b-488b-8a92-f71478fc413a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:29.584175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dc511-3e8b-488b-8a92-f71478fc413a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:29.584175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dc511-3e8b-488b-8a92-f71478fc413a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:29.604973 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 694/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:29.809781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175359e-71ae-41a3-af36-a73bfcea45fc None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:29.810316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175359e-71ae-41a3-af36-a73bfcea45fc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:29.810905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175359e-71ae-41a3-af36-a73bfcea45fc None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:29.815865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 680/1374] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:29.816821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9498ba-c2d0-4262-8ec0-ef84eaf3e10b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:29.817376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9498ba-c2d0-4262-8ec0-ef84eaf3e10b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:29.817790 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9498ba-c2d0-4262-8ec0-ef84eaf3e10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:29.898491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0d9498ba-c2d0-4262-8ec0-ef84eaf3e10b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:29.899830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0d9498ba-c2d0-4262-8ec0-ef84eaf3e10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:29.927390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:30.004197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 695/1375] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:30.048645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175359e-71ae-41a3-af36-a73bfcea45fc None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:30.050188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b175359e-71ae-41a3-af36-a73bfcea45fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:30.194928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175359e-71ae-41a3-af36-a73bfcea45fc None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:30.248849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:30.311994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3c1b8-17e7-425e-aba7-032f20041ab6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:30.312639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3c1b8-17e7-425e-aba7-032f20041ab6 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:30.312919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3c1b8-17e7-425e-aba7-032f20041ab6 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:30.313192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3c1b8-17e7-425e-aba7-032f20041ab6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:30.314861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-27f3c1b8-17e7-425e-aba7-032f20041ab6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:30.407512 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3c1b8-17e7-425e-aba7-032f20041ab6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:30.409341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3c1b8-17e7-425e-aba7-032f20041ab6 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:30.428014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 681/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:32.640686 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b9823-d046-41fe-add9-4c0a98ee39f1 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:32.641200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b9823-d046-41fe-add9-4c0a98ee39f1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:32.641329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b9823-d046-41fe-add9-4c0a98ee39f1 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:32.642639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 696/1377] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:32.647766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36018849-49e0-4da3-a6f5-25f3aaf4c9fd None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:32.648202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36018849-49e0-4da3-a6f5-25f3aaf4c9fd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:32.648328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36018849-49e0-4da3-a6f5-25f3aaf4c9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:32.731994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-36018849-49e0-4da3-a6f5-25f3aaf4c9fd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:32.733194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-36018849-49e0-4da3-a6f5-25f3aaf4c9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:32.749248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:32.865555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 682/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:32.910246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b9823-d046-41fe-add9-4c0a98ee39f1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:32.911726 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1b7b9823-d046-41fe-add9-4c0a98ee39f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:33.057780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b9823-d046-41fe-add9-4c0a98ee39f1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:33.061353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:33.188830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd73c6-f93a-4776-b513-1bf9bd449dba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:33.189401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd73c6-f93a-4776-b513-1bf9bd449dba service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:33.189740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd73c6-f93a-4776-b513-1bf9bd449dba service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:33.190024 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd73c6-f93a-4776-b513-1bf9bd449dba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:33.194550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d6fd73c6-f93a-4776-b513-1bf9bd449dba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:33.322839 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd73c6-f93a-4776-b513-1bf9bd449dba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:33.325367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd73c6-f93a-4776-b513-1bf9bd449dba service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:33.346611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 697/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:35.550298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea89b2-3aef-49a3-9469-295bc2abcff2 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:35.550866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea89b2-3aef-49a3-9469-295bc2abcff2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:35.550866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea89b2-3aef-49a3-9469-295bc2abcff2 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:35.552351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 683/1380] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:35.558234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c05afb-cc25-4e65-a11c-007ff94e730d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:35.559017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c05afb-cc25-4e65-a11c-007ff94e730d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:35.559017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c05afb-cc25-4e65-a11c-007ff94e730d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:35.570259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:35.639377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-97c05afb-cc25-4e65-a11c-007ff94e730d None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:35.639377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-97c05afb-cc25-4e65-a11c-007ff94e730d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:35.741365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 698/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:35.786010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea89b2-3aef-49a3-9469-295bc2abcff2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:35.787580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-55ea89b2-3aef-49a3-9469-295bc2abcff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:35.966605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:35.966605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea89b2-3aef-49a3-9469-295bc2abcff2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:36.085311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7676bfd-36c2-4bf5-aca1-892a69b65376 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:36.086151 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7676bfd-36c2-4bf5-aca1-892a69b65376 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:36.086430 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7676bfd-36c2-4bf5-aca1-892a69b65376 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:36.086699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7676bfd-36c2-4bf5-aca1-892a69b65376 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:36.088647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f7676bfd-36c2-4bf5-aca1-892a69b65376 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:36.192187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7676bfd-36c2-4bf5-aca1-892a69b65376 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:36.193741 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7676bfd-36c2-4bf5-aca1-892a69b65376 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:36.204864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:36.213251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 684/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:38.492329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f2f8-4893-43fb-a3b6-05a8b2581176 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:38.492962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f2f8-4893-43fb-a3b6-05a8b2581176 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:38.492962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f2f8-4893-43fb-a3b6-05a8b2581176 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:38.494764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 699/1383] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:38.500408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758885af-f44b-4da3-84e4-fc92d1b5d354 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:38.500853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758885af-f44b-4da3-84e4-fc92d1b5d354 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:38.501145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758885af-f44b-4da3-84e4-fc92d1b5d354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:38.576184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-758885af-f44b-4da3-84e4-fc92d1b5d354 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:38.577557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-758885af-f44b-4da3-84e4-fc92d1b5d354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:38.690053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 685/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:38.733711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3f2f8-4893-43fb-a3b6-05a8b2581176 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:38.735195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6fd3f2f8-4893-43fb-a3b6-05a8b2581176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:38.840391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:38.907137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3f2f8-4893-43fb-a3b6-05a8b2581176 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:39.022769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06c1d8-2aa0-49be-9e74-5a1fb234cb0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:39.023342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06c1d8-2aa0-49be-9e74-5a1fb234cb0b service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:39.023624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06c1d8-2aa0-49be-9e74-5a1fb234cb0b service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:39.023954 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06c1d8-2aa0-49be-9e74-5a1fb234cb0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:39.025592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7b06c1d8-2aa0-49be-9e74-5a1fb234cb0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:39.076309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:39.120137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06c1d8-2aa0-49be-9e74-5a1fb234cb0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:39.121920 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06c1d8-2aa0-49be-9e74-5a1fb234cb0b service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:39.145139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 700/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:41.413594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4b952-aa9c-46ba-87ae-ae868b6fa0d3 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:41.414172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4b952-aa9c-46ba-87ae-ae868b6fa0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:41.414274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4b952-aa9c-46ba-87ae-ae868b6fa0d3 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:41.416146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 686/1386] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:41.422186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91aa9a-b3c2-44d9-804d-10ddeb22fdd3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:41.422642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91aa9a-b3c2-44d9-804d-10ddeb22fdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:41.422835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91aa9a-b3c2-44d9-804d-10ddeb22fdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:41.504122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ef91aa9a-b3c2-44d9-804d-10ddeb22fdd3 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:41.506171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ef91aa9a-b3c2-44d9-804d-10ddeb22fdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:41.617426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 701/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:41.663417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4b952-aa9c-46ba-87ae-ae868b6fa0d3 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:41.665390 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3af4b952-aa9c-46ba-87ae-ae868b6fa0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:41.751740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:41.842193 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4b952-aa9c-46ba-87ae-ae868b6fa0d3 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:41.979362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ad394-a783-4102-b72e-4b1ab8ed33b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:41.980042 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ad394-a783-4102-b72e-4b1ab8ed33b8 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:41.980322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ad394-a783-4102-b72e-4b1ab8ed33b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:41.980597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ad394-a783-4102-b72e-4b1ab8ed33b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:41.982313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d25ad394-a783-4102-b72e-4b1ab8ed33b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:42.023287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:42.080863 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ad394-a783-4102-b72e-4b1ab8ed33b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:42.082495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ad394-a783-4102-b72e-4b1ab8ed33b8 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:42.101576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 687/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:44.280028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d72b11-8c64-4142-b8d8-f842f416693a None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:44.280566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d72b11-8c64-4142-b8d8-f842f416693a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:44.280656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d72b11-8c64-4142-b8d8-f842f416693a None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:44.281999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 702/1389] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:44.287527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8b6fc-fbd2-400d-b149-f85a499610af None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:44.287960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8b6fc-fbd2-400d-b149-f85a499610af None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:44.288203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8b6fc-fbd2-400d-b149-f85a499610af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:44.352637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-42a8b6fc-fbd2-400d-b149-f85a499610af None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:44.354423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-42a8b6fc-fbd2-400d-b149-f85a499610af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:44.449537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 688/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:44.493009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d72b11-8c64-4142-b8d8-f842f416693a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:44.495162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-06d72b11-8c64-4142-b8d8-f842f416693a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:44.525826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:44.640017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d72b11-8c64-4142-b8d8-f842f416693a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:44.743837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:44.755253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42699b2-5cef-4f4d-b1be-5983ea1a7f76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:44.758361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42699b2-5cef-4f4d-b1be-5983ea1a7f76 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:44.758361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42699b2-5cef-4f4d-b1be-5983ea1a7f76 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:44.758361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42699b2-5cef-4f4d-b1be-5983ea1a7f76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:44.758361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e42699b2-5cef-4f4d-b1be-5983ea1a7f76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:44.848931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42699b2-5cef-4f4d-b1be-5983ea1a7f76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:44.850574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42699b2-5cef-4f4d-b1be-5983ea1a7f76 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:44.870051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 703/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:45.100538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f55b3-c19f-4852-9d65-cba19a320cb5 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:45.100851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f55b3-c19f-4852-9d65-cba19a320cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:45.101133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f55b3-c19f-4852-9d65-cba19a320cb5 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:45.102588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 689/1392] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:45.108770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6c5b6-d797-46bc-acad-6bd5b59e9b4d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:45.109056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6c5b6-d797-46bc-acad-6bd5b59e9b4d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:45.109338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6c5b6-d797-46bc-acad-6bd5b59e9b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:45.182672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-21f6c5b6-d797-46bc-acad-6bd5b59e9b4d None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:45.183916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-21f6c5b6-d797-46bc-acad-6bd5b59e9b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:45.245303 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:45.284532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 704/1393] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:45.339649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fcac6-f4ad-481f-a362-b0b4e83c4d48 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:45.340139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fcac6-f4ad-481f-a362-b0b4e83c4d48 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:45.340256 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fcac6-f4ad-481f-a362-b0b4e83c4d48 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:45.342228 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 690/1394] 10.222.0.40 () {54 vars in 962 bytes} [Wed Jul 13 08:58:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:45.347635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4af1f-4923-4a8d-8d48-9694e793e3b9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:45.347919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4af1f-4923-4a8d-8d48-9694e793e3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:45.348192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4af1f-4923-4a8d-8d48-9694e793e3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:45.422936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fde4af1f-4923-4a8d-8d48-9694e793e3b9 None None] MFA Rules not processed for user `cf67fa55f4d64b7ebeeb1490453661e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:45.424115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fde4af1f-4923-4a8d-8d48-9694e793e3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:45.501816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 705/1395] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Jul 13 08:58:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 08:58:45.509369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fcac6-f4ad-481f-a362-b0b4e83c4d48 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:45.510881 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-819fcac6-f4ad-481f-a362-b0b4e83c4d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:45.524537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:45.616212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fcac6-f4ad-481f-a362-b0b4e83c4d48 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:45.690118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bd16b-d754-4fee-bb0c-02f3f44c77d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf67fa55f4d64b7ebeeb1490453661e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:45.690769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd16b-d754-4fee-bb0c-02f3f44c77d2 service glance] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:45.691053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd16b-d754-4fee-bb0c-02f3f44c77d2 service glance] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:45.691332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd16b-d754-4fee-bb0c-02f3f44c77d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:45.692999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-484bd16b-d754-4fee-bb0c-02f3f44c77d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:45.765445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:45.807917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bd16b-d754-4fee-bb0c-02f3f44c77d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:45.809333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bd16b-d754-4fee-bb0c-02f3f44c77d2 service glance] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:45.828286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 691/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:46.700636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ecd23-efe1-4ea7-8b5a-84cb5a2251cd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:46.701170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ecd23-efe1-4ea7-8b5a-84cb5a2251cd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:46.701260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ecd23-efe1-4ea7-8b5a-84cb5a2251cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:46.769423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-633ecd23-efe1-4ea7-8b5a-84cb5a2251cd None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:46.770710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-633ecd23-efe1-4ea7-8b5a-84cb5a2251cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:46.780069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:46.859566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 706/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 08:58:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:46.906503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7436e-b1b2-4544-a6ef-d40f38598421 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:46.907004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7436e-b1b2-4544-a6ef-d40f38598421 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:46.907121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7436e-b1b2-4544-a6ef-d40f38598421 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:46.908744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 692/1398] 10.222.0.40 () {54 vars in 942 bytes} [Wed Jul 13 08:58:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:46.915212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe81484-a055-46a1-bbdf-3b66dc29ae9b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:46.915599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe81484-a055-46a1-bbdf-3b66dc29ae9b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:46.916027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe81484-a055-46a1-bbdf-3b66dc29ae9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:46.991277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1fe81484-a055-46a1-bbdf-3b66dc29ae9b None None] MFA Rules not processed for user `906fc971a5ee4ffd9503ab13d46cd605`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:46.992490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1fe81484-a055-46a1-bbdf-3b66dc29ae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:47.079961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:47.092831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750dd39-d58d-444c-a0d6-64b7e7ccb463 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:47.093112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750dd39-d58d-444c-a0d6-64b7e7ccb463 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:47.093395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750dd39-d58d-444c-a0d6-64b7e7ccb463 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:47.094884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 693/1399] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:47.099703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d8ddf-f7fe-43d7-997e-d1cd3eb69f4a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:47.099991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d8ddf-f7fe-43d7-997e-d1cd3eb69f4a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:47.100269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d8ddf-f7fe-43d7-997e-d1cd3eb69f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:47.103063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 707/1400] 10.222.0.40 () {58 vars in 1039 bytes} [Wed Jul 13 08:58:46 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:47.113143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe81484-a055-46a1-bbdf-3b66dc29ae9b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:47.123453 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe81484-a055-46a1-bbdf-3b66dc29ae9b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:47.166838 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-038d8ddf-f7fe-43d7-997e-d1cd3eb69f4a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:47.168037 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-038d8ddf-f7fe-43d7-997e-d1cd3eb69f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:47.229329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab6c14-ac0c-462a-8599-6756621586b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:47.229968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab6c14-ac0c-462a-8599-6756621586b9 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:47.230273 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab6c14-ac0c-462a-8599-6756621586b9 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:47.230545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab6c14-ac0c-462a-8599-6756621586b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:47.232206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7aab6c14-ac0c-462a-8599-6756621586b9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:47.266444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 694/1401] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 08:58:47.312180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d8ddf-f7fe-43d7-997e-d1cd3eb69f4a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:47.313358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-038d8ddf-f7fe-43d7-997e-d1cd3eb69f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:47.319210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab6c14-ac0c-462a-8599-6756621586b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:47.320901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab6c14-ac0c-462a-8599-6756621586b9 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:47.329882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 708/1402] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 08:58:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 08:58:47.340753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:47.429532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d8ddf-f7fe-43d7-997e-d1cd3eb69f4a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:47.543133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068c020-c236-4da7-96e3-39dd2dc5c663 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:47.543778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068c020-c236-4da7-96e3-39dd2dc5c663 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:47.544064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068c020-c236-4da7-96e3-39dd2dc5c663 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:47.544337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068c020-c236-4da7-96e3-39dd2dc5c663 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:47.545973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0068c020-c236-4da7-96e3-39dd2dc5c663 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:47.564128 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:47.640795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068c020-c236-4da7-96e3-39dd2dc5c663 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:47.642471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068c020-c236-4da7-96e3-39dd2dc5c663 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:47.660745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 695/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 08:58:49.920430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795403bf-1fa6-4d5c-8f17-dfddeea9813e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:49.921028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795403bf-1fa6-4d5c-8f17-dfddeea9813e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:49.921028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795403bf-1fa6-4d5c-8f17-dfddeea9813e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:49.922379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 709/1404] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:49.927586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59c73-909d-44ae-b282-8b1b603a5a36 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:49.928003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59c73-909d-44ae-b282-8b1b603a5a36 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:49.928156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59c73-909d-44ae-b282-8b1b603a5a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:49.997241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-35f59c73-909d-44ae-b282-8b1b603a5a36 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:49.998482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-35f59c73-909d-44ae-b282-8b1b603a5a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:50.099817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:50.107706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 696/1405] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:50.155335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795403bf-1fa6-4d5c-8f17-dfddeea9813e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:50.157003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-795403bf-1fa6-4d5c-8f17-dfddeea9813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:50.168437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:50.318142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795403bf-1fa6-4d5c-8f17-dfddeea9813e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:50.417148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:50.451908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1411f6-7141-456b-8815-f2dca6a4e70c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:50.452837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1411f6-7141-456b-8815-f2dca6a4e70c service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:50.453117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1411f6-7141-456b-8815-f2dca6a4e70c service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:50.453390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1411f6-7141-456b-8815-f2dca6a4e70c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:50.455072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5d1411f6-7141-456b-8815-f2dca6a4e70c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:50.563554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1411f6-7141-456b-8815-f2dca6a4e70c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:50.565156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1411f6-7141-456b-8815-f2dca6a4e70c service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:50.599801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 710/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:58:53.423876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ffb74-189a-4632-8ddc-ae2d16627aa6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:53.424459 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ffb74-189a-4632-8ddc-ae2d16627aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:53.424552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ffb74-189a-4632-8ddc-ae2d16627aa6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:53.426004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 697/1407] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 08:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:58:53.431622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81975a-65b6-4268-bb10-f5c7ee458e2a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:53.432063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81975a-65b6-4268-bb10-f5c7ee458e2a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:53.432166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81975a-65b6-4268-bb10-f5c7ee458e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:53.501569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3d81975a-65b6-4268-bb10-f5c7ee458e2a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:58:53.503140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3d81975a-65b6-4268-bb10-f5c7ee458e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:53.558116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:53.621684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 711/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 08:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:58:53.667217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ffb74-189a-4632-8ddc-ae2d16627aa6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:58:53.668718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6d0ffb74-189a-4632-8ddc-ae2d16627aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:53.839484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ffb74-189a-4632-8ddc-ae2d16627aa6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:58:53.917219 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:58:53.969096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb6d6c-50f6-4da6-99c7-bc4e6f4517d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:58:53.970205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb6d6c-50f6-4da6-99c7-bc4e6f4517d9 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:58:53.970700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb6d6c-50f6-4da6-99c7-bc4e6f4517d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:58:53.971264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb6d6c-50f6-4da6-99c7-bc4e6f4517d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:58:53.974329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-05cb6d6c-50f6-4da6-99c7-bc4e6f4517d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:58:54.070385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb6d6c-50f6-4da6-99c7-bc4e6f4517d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:58:54.072778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb6d6c-50f6-4da6-99c7-bc4e6f4517d9 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:58:54.092580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 698/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:03.619613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=23 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:04.090525 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=64 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:10.973003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27b9c2-6d87-46db-901f-cf00d3af1796 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:10.973574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27b9c2-6d87-46db-901f-cf00d3af1796 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:10.973574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27b9c2-6d87-46db-901f-cf00d3af1796 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:10.974936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 712/1410] 10.222.0.40 () {54 vars in 918 bytes} [Wed Jul 13 08:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:10.980923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbc1a8-6a0f-480d-924d-bd74aa1f7b8a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:10.981380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbc1a8-6a0f-480d-924d-bd74aa1f7b8a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:10.981489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbc1a8-6a0f-480d-924d-bd74aa1f7b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:11.057691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3abbc1a8-6a0f-480d-924d-bd74aa1f7b8a None None] MFA Rules not processed for user `bca7c1164d3e4833a10ffc37cbc6f092`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:11.058932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3abbc1a8-6a0f-480d-924d-bd74aa1f7b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:11.164252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 699/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Jul 13 08:59:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:11.179104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796ad55-2031-4035-8f51-8c55de9c09bb None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:11.179388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796ad55-2031-4035-8f51-8c55de9c09bb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:11.179661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796ad55-2031-4035-8f51-8c55de9c09bb None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:11.181073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 713/1412] 10.222.0.40 () {54 vars in 918 bytes} [Wed Jul 13 08:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:11.185779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e1480-9e41-431f-b6eb-86a933a1f5a0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:11.186089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e1480-9e41-431f-b6eb-86a933a1f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:11.186367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e1480-9e41-431f-b6eb-86a933a1f5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:11.256183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:11.258816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5d2e1480-9e41-431f-b6eb-86a933a1f5a0 None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:11.260006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5d2e1480-9e41-431f-b6eb-86a933a1f5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:11.378001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 700/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Jul 13 08:59:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:13.876985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cec162-2f00-4e62-b168-96d338431021 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:13.877560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cec162-2f00-4e62-b168-96d338431021 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:13.877560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cec162-2f00-4e62-b168-96d338431021 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:13.878954 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:13.884329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5853b9-f54d-447e-acfc-ca5051bfef2b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:13.884777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5853b9-f54d-447e-acfc-ca5051bfef2b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:13.884887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5853b9-f54d-447e-acfc-ca5051bfef2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:13.949340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0c5853b9-f54d-447e-acfc-ca5051bfef2b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:13.950565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0c5853b9-f54d-447e-acfc-ca5051bfef2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:14.006545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:14.048811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:14.090390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cec162-2f00-4e62-b168-96d338431021 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:14.091948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e1cec162-2f00-4e62-b168-96d338431021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:14.239262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cec162-2f00-4e62-b168-96d338431021 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:14.310125 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:14.349982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1ae6d-c366-49eb-bc8a-34c00162b619 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:14.350645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ae6d-c366-49eb-bc8a-34c00162b619 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:14.350924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ae6d-c366-49eb-bc8a-34c00162b619 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:14.351193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ae6d-c366-49eb-bc8a-34c00162b619 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:14.353570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-24b1ae6d-c366-49eb-bc8a-34c00162b619 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:14.440812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1ae6d-c366-49eb-bc8a-34c00162b619 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:14.442418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1ae6d-c366-49eb-bc8a-34c00162b619 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:14.459874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 715/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:15.362993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522364da-a0a6-4610-98be-6471f3c88390 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:15.363436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522364da-a0a6-4610-98be-6471f3c88390 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:15.363542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522364da-a0a6-4610-98be-6471f3c88390 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:15.364883 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 08:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:15.370833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548da78-bd20-497e-9c3f-35cd2d41c4f9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:15.371270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548da78-bd20-497e-9c3f-35cd2d41c4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:15.371388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548da78-bd20-497e-9c3f-35cd2d41c4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:15.445156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f548da78-bd20-497e-9c3f-35cd2d41c4f9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:15.446412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f548da78-bd20-497e-9c3f-35cd2d41c4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:15.473271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:15.545708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 08:59:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:15.697572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522364da-a0a6-4610-98be-6471f3c88390 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:15.699111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-522364da-a0a6-4610-98be-6471f3c88390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:15.844074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522364da-a0a6-4610-98be-6471f3c88390 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:15.883787 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:15.958045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae37877-4f4c-42c8-8091-fe5f0edd228a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:15.958630 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae37877-4f4c-42c8-8091-fe5f0edd228a service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:15.958909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae37877-4f4c-42c8-8091-fe5f0edd228a service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:15.959182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae37877-4f4c-42c8-8091-fe5f0edd228a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:15.960801 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6ae37877-4f4c-42c8-8091-fe5f0edd228a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:16.049237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae37877-4f4c-42c8-8091-fe5f0edd228a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:16.050852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae37877-4f4c-42c8-8091-fe5f0edd228a service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:16.068941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 703/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:16.949565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4d66-7fb4-4447-a38e-d8c9f486dfb5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:16.950145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4d66-7fb4-4447-a38e-d8c9f486dfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:16.950145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4d66-7fb4-4447-a38e-d8c9f486dfb5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:16.951571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 08:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:16.956412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591dcda-ba73-4ec8-b9cd-298aa4049a36 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:16.956839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591dcda-ba73-4ec8-b9cd-298aa4049a36 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:16.957061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591dcda-ba73-4ec8-b9cd-298aa4049a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:17.011022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:17.033251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7591dcda-ba73-4ec8-b9cd-298aa4049a36 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:17.034505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7591dcda-ba73-4ec8-b9cd-298aa4049a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:17.136932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 08:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:17.296126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4d66-7fb4-4447-a38e-d8c9f486dfb5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:17.297751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3bcd4d66-7fb4-4447-a38e-d8c9f486dfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:17.455553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4d66-7fb4-4447-a38e-d8c9f486dfb5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:17.468713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:17.587856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32432a-96ba-48ac-8a8a-32748ee4273d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:17.588404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32432a-96ba-48ac-8a8a-32748ee4273d service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:17.588681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32432a-96ba-48ac-8a8a-32748ee4273d service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:17.588951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32432a-96ba-48ac-8a8a-32748ee4273d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:17.590700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2f32432a-96ba-48ac-8a8a-32748ee4273d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:17.688966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32432a-96ba-48ac-8a8a-32748ee4273d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:17.690712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32432a-96ba-48ac-8a8a-32748ee4273d service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:17.713182 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 718/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:18.596184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35013e7a-ebe2-447b-9e64-896dfd8a53c5 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:18.596699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35013e7a-ebe2-447b-9e64-896dfd8a53c5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:18.596699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35013e7a-ebe2-447b-9e64-896dfd8a53c5 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:18.598001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 08:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:18.603583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa54d6-3a2e-4f4c-a4a5-fe10df8bd80e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:18.603988 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa54d6-3a2e-4f4c-a4a5-fe10df8bd80e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:18.604166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa54d6-3a2e-4f4c-a4a5-fe10df8bd80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:18.622901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:18.679156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-cffa54d6-3a2e-4f4c-a4a5-fe10df8bd80e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:18.680346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cffa54d6-3a2e-4f4c-a4a5-fe10df8bd80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:18.775467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 08:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:18.925652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35013e7a-ebe2-447b-9e64-896dfd8a53c5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:18.927454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-35013e7a-ebe2-447b-9e64-896dfd8a53c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:19.037380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:19.073370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35013e7a-ebe2-447b-9e64-896dfd8a53c5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:19.185047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e93f5-d7d5-4cef-9997-2b8dec8c85af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:19.185616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e93f5-d7d5-4cef-9997-2b8dec8c85af service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:19.185891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e93f5-d7d5-4cef-9997-2b8dec8c85af service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:19.186189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e93f5-d7d5-4cef-9997-2b8dec8c85af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:19.187809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b95e93f5-d7d5-4cef-9997-2b8dec8c85af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:19.263515 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:19.277375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e93f5-d7d5-4cef-9997-2b8dec8c85af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:19.279004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e93f5-d7d5-4cef-9997-2b8dec8c85af service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:19.297867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 706/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:20.149618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72e711-daf8-4527-8a59-7f1e625161c2 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:20.150190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72e711-daf8-4527-8a59-7f1e625161c2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:20.150190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72e711-daf8-4527-8a59-7f1e625161c2 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:20.151607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 08:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:20.156706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c14e2-feea-430c-9c38-e90d16d079cf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:20.157404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c14e2-feea-430c-9c38-e90d16d079cf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:20.157514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c14e2-feea-430c-9c38-e90d16d079cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:20.232880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-491c14e2-feea-430c-9c38-e90d16d079cf None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:20.234111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-491c14e2-feea-430c-9c38-e90d16d079cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:20.329528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 08:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:20.482104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72e711-daf8-4527-8a59-7f1e625161c2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:20.483577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4a72e711-daf8-4527-8a59-7f1e625161c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:20.569574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:20.630986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72e711-daf8-4527-8a59-7f1e625161c2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:20.743843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f0629-038a-4dae-be2f-8d5b35945345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:20.744410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f0629-038a-4dae-be2f-8d5b35945345 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:20.744689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f0629-038a-4dae-be2f-8d5b35945345 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:20.744963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f0629-038a-4dae-be2f-8d5b35945345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:20.746630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e52f0629-038a-4dae-be2f-8d5b35945345 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:20.794888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:20.835580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f0629-038a-4dae-be2f-8d5b35945345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:20.837754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f0629-038a-4dae-be2f-8d5b35945345 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:20.856155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 721/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:21.702892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a4b97-0d77-47dc-8de4-611414dc03d4 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:21.703451 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a4b97-0d77-47dc-8de4-611414dc03d4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:21.703451 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a4b97-0d77-47dc-8de4-611414dc03d4 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:21.704811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 08:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:21.709565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66c049-ae90-4ad0-9730-a74e37115f19 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:21.710015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66c049-ae90-4ad0-9730-a74e37115f19 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:21.710178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66c049-ae90-4ad0-9730-a74e37115f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:21.785075 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6a66c049-ae90-4ad0-9730-a74e37115f19 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:21.786319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6a66c049-ae90-4ad0-9730-a74e37115f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:21.883436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 08:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:22.031314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a4b97-0d77-47dc-8de4-611414dc03d4 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:22.033128 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dc4a4b97-0d77-47dc-8de4-611414dc03d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:22.082329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:22.180218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a4b97-0d77-47dc-8de4-611414dc03d4 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:22.292417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41637f44-9772-4857-9a8b-39b7fdcbb9cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:22.292980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41637f44-9772-4857-9a8b-39b7fdcbb9cd service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:22.293258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41637f44-9772-4857-9a8b-39b7fdcbb9cd service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:22.293529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41637f44-9772-4857-9a8b-39b7fdcbb9cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:22.295195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-41637f44-9772-4857-9a8b-39b7fdcbb9cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:22.301929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:22.383652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41637f44-9772-4857-9a8b-39b7fdcbb9cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:22.385227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41637f44-9772-4857-9a8b-39b7fdcbb9cd service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:22.403191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 709/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:23.256034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee55ca-4701-48ab-9f7e-55b5c27d9194 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:23.256629 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee55ca-4701-48ab-9f7e-55b5c27d9194 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:23.256629 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee55ca-4701-48ab-9f7e-55b5c27d9194 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:23.258682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 08:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:23.263762 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321c56-4df1-43bf-aa2f-af63ee3301b4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:23.264162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321c56-4df1-43bf-aa2f-af63ee3301b4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:23.264312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321c56-4df1-43bf-aa2f-af63ee3301b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:23.338967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d3321c56-4df1-43bf-aa2f-af63ee3301b4 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:23.340181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d3321c56-4df1-43bf-aa2f-af63ee3301b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:23.407642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:23.436841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 08:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:23.585482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee55ca-4701-48ab-9f7e-55b5c27d9194 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:23.587452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-90ee55ca-4701-48ab-9f7e-55b5c27d9194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:23.605574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:23.736091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee55ca-4701-48ab-9f7e-55b5c27d9194 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:23.825752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:23.850857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f91e6-3587-4f1e-af3e-1bc99ce6a4d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:23.851535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f91e6-3587-4f1e-af3e-1bc99ce6a4d3 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:23.851820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f91e6-3587-4f1e-af3e-1bc99ce6a4d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:23.852124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f91e6-3587-4f1e-af3e-1bc99ce6a4d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:23.853793 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-382f91e6-3587-4f1e-af3e-1bc99ce6a4d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:23.942933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f91e6-3587-4f1e-af3e-1bc99ce6a4d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:23.944508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f91e6-3587-4f1e-af3e-1bc99ce6a4d3 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:23.962965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 724/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:24.814635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee0cd5-ec77-4935-889d-7b9171e53899 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:24.815188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee0cd5-ec77-4935-889d-7b9171e53899 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:24.815188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee0cd5-ec77-4935-889d-7b9171e53899 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:24.816586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 08:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:24.822326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49bd55-5a35-46bc-91a8-88678adf7dec None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:24.822751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49bd55-5a35-46bc-91a8-88678adf7dec None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:24.822856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49bd55-5a35-46bc-91a8-88678adf7dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:24.895910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1b49bd55-5a35-46bc-91a8-88678adf7dec None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:24.897096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1b49bd55-5a35-46bc-91a8-88678adf7dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:24.938612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:24.994590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 08:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 08:59:25.145997 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee0cd5-ec77-4935-889d-7b9171e53899 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:25.147548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ceee0cd5-ec77-4935-889d-7b9171e53899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:25.292357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee0cd5-ec77-4935-889d-7b9171e53899 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:25.349160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:25.411960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9971971-2ba8-4122-9406-5ca154be6f18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:25.412626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9971971-2ba8-4122-9406-5ca154be6f18 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:25.412907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9971971-2ba8-4122-9406-5ca154be6f18 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:25.413174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9971971-2ba8-4122-9406-5ca154be6f18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:25.415414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a9971971-2ba8-4122-9406-5ca154be6f18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:25.512459 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9971971-2ba8-4122-9406-5ca154be6f18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:25.514059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9971971-2ba8-4122-9406-5ca154be6f18 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:25.533042 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 712/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:26.398062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaf22c-a777-419e-b19b-bbf7fbb679ae None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:26.398573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaf22c-a777-419e-b19b-bbf7fbb679ae None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:26.398668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaf22c-a777-419e-b19b-bbf7fbb679ae None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:26.400023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 08:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:26.404861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca91f39-f7b0-431e-8c40-49d46e3e7dd8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:26.405248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca91f39-f7b0-431e-8c40-49d46e3e7dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:26.405409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca91f39-f7b0-431e-8c40-49d46e3e7dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:26.481440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-cca91f39-f7b0-431e-8c40-49d46e3e7dd8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:26.482673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cca91f39-f7b0-431e-8c40-49d46e3e7dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:26.484674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:26.581093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 08:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 08:59:26.736702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddaf22c-a777-419e-b19b-bbf7fbb679ae None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:26.738303 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1ddaf22c-a777-419e-b19b-bbf7fbb679ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:26.880338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddaf22c-a777-419e-b19b-bbf7fbb679ae None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:26.906336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:26.990649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53d511-4c02-447c-b417-c23083e5297b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:26.991234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53d511-4c02-447c-b417-c23083e5297b service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:26.991543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53d511-4c02-447c-b417-c23083e5297b service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:26.991829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53d511-4c02-447c-b417-c23083e5297b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:26.993469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1d53d511-4c02-447c-b417-c23083e5297b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:27.083550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53d511-4c02-447c-b417-c23083e5297b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:27.085143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53d511-4c02-447c-b417-c23083e5297b service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:27.103299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 727/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:29.059003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573bec4-1b27-4fb9-b6cb-1ef034f6a36a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:29.059613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573bec4-1b27-4fb9-b6cb-1ef034f6a36a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:29.059613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573bec4-1b27-4fb9-b6cb-1ef034f6a36a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:29.060918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:29.066143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31c16f-8e47-4f4c-a884-275ed757aaea None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:29.066581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31c16f-8e47-4f4c-a884-275ed757aaea None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:29.066777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31c16f-8e47-4f4c-a884-275ed757aaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:29.106897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:29.130793 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7d31c16f-8e47-4f4c-a884-275ed757aaea None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:29.131993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7d31c16f-8e47-4f4c-a884-275ed757aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:29.228039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:29.266607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573bec4-1b27-4fb9-b6cb-1ef034f6a36a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:29.268136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b573bec4-1b27-4fb9-b6cb-1ef034f6a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:29.371715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573bec4-1b27-4fb9-b6cb-1ef034f6a36a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:29.403761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:29.443982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8f0db-3cce-4a5b-8c4d-22e991605431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:29.444615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8f0db-3cce-4a5b-8c4d-22e991605431 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:29.444896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8f0db-3cce-4a5b-8c4d-22e991605431 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:29.445161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8f0db-3cce-4a5b-8c4d-22e991605431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:29.447310 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f6a8f0db-3cce-4a5b-8c4d-22e991605431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:29.554818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8f0db-3cce-4a5b-8c4d-22e991605431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:29.556402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8f0db-3cce-4a5b-8c4d-22e991605431 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:29.575302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 715/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:31.755961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803859-87f7-4403-824d-8813c9f39e4e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:31.756530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803859-87f7-4403-824d-8813c9f39e4e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:31.756530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803859-87f7-4403-824d-8813c9f39e4e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:31.757891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:31.763176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3a05-9443-4016-90ee-e1bc25bd288e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:31.763609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3a05-9443-4016-90ee-e1bc25bd288e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:31.763736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3a05-9443-4016-90ee-e1bc25bd288e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:31.830158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c8bc3a05-9443-4016-90ee-e1bc25bd288e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:31.831352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c8bc3a05-9443-4016-90ee-e1bc25bd288e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:31.844351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:31.928455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:31.966162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803859-87f7-4403-824d-8813c9f39e4e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:31.967677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6a803859-87f7-4403-824d-8813c9f39e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:32.075367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803859-87f7-4403-824d-8813c9f39e4e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:32.087879 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:32.148937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c67b9-67bc-4c3a-b052-c9dbe1bb93ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:32.149620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c67b9-67bc-4c3a-b052-c9dbe1bb93ca service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:32.150561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c67b9-67bc-4c3a-b052-c9dbe1bb93ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:32.151018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c67b9-67bc-4c3a-b052-c9dbe1bb93ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:32.152583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-941c67b9-67bc-4c3a-b052-c9dbe1bb93ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:32.261760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c67b9-67bc-4c3a-b052-c9dbe1bb93ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:32.263170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c67b9-67bc-4c3a-b052-c9dbe1bb93ca service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:32.283263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 730/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:34.702124 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e0106-8fce-4517-97da-6bb16e1fd290 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:34.702693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e0106-8fce-4517-97da-6bb16e1fd290 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:34.702693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e0106-8fce-4517-97da-6bb16e1fd290 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:34.704035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:34.709392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cba611-a911-4e8d-a685-2ca1af3dd41d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:34.709812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cba611-a911-4e8d-a685-2ca1af3dd41d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:34.709960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cba611-a911-4e8d-a685-2ca1af3dd41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:34.757930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:34.775891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-18cba611-a911-4e8d-a685-2ca1af3dd41d None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:34.777087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-18cba611-a911-4e8d-a685-2ca1af3dd41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:34.874928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:34.914667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e0106-8fce-4517-97da-6bb16e1fd290 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:34.916811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-580e0106-8fce-4517-97da-6bb16e1fd290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:35.018818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e0106-8fce-4517-97da-6bb16e1fd290 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:35.059522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:35.093938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8d904-ce62-419d-95cf-13b851bcc866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:35.094610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8d904-ce62-419d-95cf-13b851bcc866 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:35.094894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8d904-ce62-419d-95cf-13b851bcc866 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:35.095171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8d904-ce62-419d-95cf-13b851bcc866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:35.096806 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9ae8d904-ce62-419d-95cf-13b851bcc866 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:35.204077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8d904-ce62-419d-95cf-13b851bcc866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:35.205496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8d904-ce62-419d-95cf-13b851bcc866 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:35.224108 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 718/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:37.358495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6346c9-9872-4333-ac17-44ff7784d4cd None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:37.358937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6346c9-9872-4333-ac17-44ff7784d4cd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:37.359051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6346c9-9872-4333-ac17-44ff7784d4cd None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:37.360399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:37.365557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84283e17-6ddf-46c2-8d9f-cc555d3c14bd None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:37.366080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84283e17-6ddf-46c2-8d9f-cc555d3c14bd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:37.366189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84283e17-6ddf-46c2-8d9f-cc555d3c14bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:37.440442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-84283e17-6ddf-46c2-8d9f-cc555d3c14bd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:37.441665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-84283e17-6ddf-46c2-8d9f-cc555d3c14bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:37.463333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:37.541616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:37.582947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6346c9-9872-4333-ac17-44ff7784d4cd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:37.584564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3c6346c9-9872-4333-ac17-44ff7784d4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:37.689064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6346c9-9872-4333-ac17-44ff7784d4cd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:37.709359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:37.769524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0d0c6-9508-4351-b1fa-13b46146a288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:37.770188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0d0c6-9508-4351-b1fa-13b46146a288 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:37.770472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0d0c6-9508-4351-b1fa-13b46146a288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:37.770748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0d0c6-9508-4351-b1fa-13b46146a288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:37.772370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cfb0d0c6-9508-4351-b1fa-13b46146a288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:37.881294 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0d0c6-9508-4351-b1fa-13b46146a288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:37.882762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0d0c6-9508-4351-b1fa-13b46146a288 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:37.901326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 733/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:39.982538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95693a-17d7-45ea-aada-a0a002c25d27 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:39.983100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95693a-17d7-45ea-aada-a0a002c25d27 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:39.983100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95693a-17d7-45ea-aada-a0a002c25d27 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:39.984418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:39.989718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a1259-4762-40e4-b8fd-3563bb63eec6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:39.990149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a1259-4762-40e4-b8fd-3563bb63eec6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:39.990300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a1259-4762-40e4-b8fd-3563bb63eec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:40.052986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:40.055632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-507a1259-4762-40e4-b8fd-3563bb63eec6 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:40.056815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-507a1259-4762-40e4-b8fd-3563bb63eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:40.152797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:40.190789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95693a-17d7-45ea-aada-a0a002c25d27 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:40.192338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cc95693a-17d7-45ea-aada-a0a002c25d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:40.296776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95693a-17d7-45ea-aada-a0a002c25d27 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:40.345642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:40.371245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe402fd-8cce-4d5a-82c2-22914f49dcfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:40.371890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe402fd-8cce-4d5a-82c2-22914f49dcfe service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:40.372169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe402fd-8cce-4d5a-82c2-22914f49dcfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:40.372512 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe402fd-8cce-4d5a-82c2-22914f49dcfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:40.374818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3fe402fd-8cce-4d5a-82c2-22914f49dcfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:40.486013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe402fd-8cce-4d5a-82c2-22914f49dcfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:40.487476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe402fd-8cce-4d5a-82c2-22914f49dcfe service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:40.505681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 721/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:42.579189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc7d60-9933-4d4e-af2e-add7021c47f4 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:42.579765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc7d60-9933-4d4e-af2e-add7021c47f4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:42.579765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc7d60-9933-4d4e-af2e-add7021c47f4 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:42.581151 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:42.586354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389eb46-a72d-488d-a6f7-d6b973b1cc05 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:42.587325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389eb46-a72d-488d-a6f7-d6b973b1cc05 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:42.591952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389eb46-a72d-488d-a6f7-d6b973b1cc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:42.652718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9389eb46-a72d-488d-a6f7-d6b973b1cc05 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:42.653921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9389eb46-a72d-488d-a6f7-d6b973b1cc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:42.684092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:42.752954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:42.791405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc7d60-9933-4d4e-af2e-add7021c47f4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:42.792996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-93fc7d60-9933-4d4e-af2e-add7021c47f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:42.894496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc7d60-9933-4d4e-af2e-add7021c47f4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:42.922675 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:42.967286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e0ea0-ab04-468e-b4d1-aab1c48f8026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:42.967918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e0ea0-ab04-468e-b4d1-aab1c48f8026 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:42.968198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e0ea0-ab04-468e-b4d1-aab1c48f8026 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:42.968470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e0ea0-ab04-468e-b4d1-aab1c48f8026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:42.970119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a72e0ea0-ab04-468e-b4d1-aab1c48f8026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:43.076666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e0ea0-ab04-468e-b4d1-aab1c48f8026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:43.078119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e0ea0-ab04-468e-b4d1-aab1c48f8026 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:43.096139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 736/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:45.181173 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02a3db-6385-4e04-8e68-6efa1a1f3234 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:45.181753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02a3db-6385-4e04-8e68-6efa1a1f3234 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:45.181753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02a3db-6385-4e04-8e68-6efa1a1f3234 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:45.183118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:45.188276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc940-31e4-4014-b5ef-cada45d3d393 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:45.188657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc940-31e4-4014-b5ef-cada45d3d393 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:45.188835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc940-31e4-4014-b5ef-cada45d3d393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:45.254355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-64dcc940-31e4-4014-b5ef-cada45d3d393 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:45.255560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-64dcc940-31e4-4014-b5ef-cada45d3d393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:45.263872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:45.359883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:45.397775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02a3db-6385-4e04-8e68-6efa1a1f3234 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:45.399884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3e02a3db-6385-4e04-8e68-6efa1a1f3234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:45.501361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02a3db-6385-4e04-8e68-6efa1a1f3234 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:45.556195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:45.574250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb3070-08a2-42bb-bc36-c427e94420d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:45.574890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb3070-08a2-42bb-bc36-c427e94420d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:45.575169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb3070-08a2-42bb-bc36-c427e94420d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:45.575440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb3070-08a2-42bb-bc36-c427e94420d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:45.577059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bcfb3070-08a2-42bb-bc36-c427e94420d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:45.684249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb3070-08a2-42bb-bc36-c427e94420d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:45.685713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb3070-08a2-42bb-bc36-c427e94420d4 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:45.703713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 724/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:47.760857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567f29-ad8c-423b-b67d-c5eedfd14c45 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:47.761432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567f29-ad8c-423b-b67d-c5eedfd14c45 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:47.761432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567f29-ad8c-423b-b67d-c5eedfd14c45 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:47.762796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:47.768002 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa785b8f-efab-45bc-acdb-a8e8561b7674 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:47.768413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa785b8f-efab-45bc-acdb-a8e8561b7674 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:47.768561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa785b8f-efab-45bc-acdb-a8e8561b7674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:47.833345 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-aa785b8f-efab-45bc-acdb-a8e8561b7674 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:47.834577 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-aa785b8f-efab-45bc-acdb-a8e8561b7674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:47.874141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:47.932612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:47.969490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84567f29-ad8c-423b-b67d-c5eedfd14c45 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:47.971075 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-84567f29-ad8c-423b-b67d-c5eedfd14c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:48.074529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84567f29-ad8c-423b-b67d-c5eedfd14c45 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:48.111661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:48.147348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c405f-8d14-419a-a918-b8e80a2392eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:48.147985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c405f-8d14-419a-a918-b8e80a2392eb service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:48.148265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c405f-8d14-419a-a918-b8e80a2392eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:48.148537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c405f-8d14-419a-a918-b8e80a2392eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:48.150837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d59c405f-8d14-419a-a918-b8e80a2392eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:48.258656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c405f-8d14-419a-a918-b8e80a2392eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:48.260293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c405f-8d14-419a-a918-b8e80a2392eb service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:48.278340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 739/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:50.352221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec76b1-baef-4832-9182-1c3cb95bb3a9 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:50.352901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec76b1-baef-4832-9182-1c3cb95bb3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:50.352901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec76b1-baef-4832-9182-1c3cb95bb3a9 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:50.354381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:50.359807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3a886-f4e3-4d85-98aa-b2000b7f5e2a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:50.360243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3a886-f4e3-4d85-98aa-b2000b7f5e2a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:50.360339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3a886-f4e3-4d85-98aa-b2000b7f5e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:50.426511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3aa3a886-f4e3-4d85-98aa-b2000b7f5e2a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:50.427752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3aa3a886-f4e3-4d85-98aa-b2000b7f5e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:50.445664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:50.525871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:50.562841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec76b1-baef-4832-9182-1c3cb95bb3a9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:50.564474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f8ec76b1-baef-4832-9182-1c3cb95bb3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:50.671892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec76b1-baef-4832-9182-1c3cb95bb3a9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:50.737982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:50.745155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374601d5-6f22-46e8-889c-e1c990b0a41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:50.749805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374601d5-6f22-46e8-889c-e1c990b0a41d service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:50.749805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374601d5-6f22-46e8-889c-e1c990b0a41d service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:50.749805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374601d5-6f22-46e8-889c-e1c990b0a41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:50.749805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-374601d5-6f22-46e8-889c-e1c990b0a41d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:50.858301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374601d5-6f22-46e8-889c-e1c990b0a41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:50.859818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374601d5-6f22-46e8-889c-e1c990b0a41d service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:50.878608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 727/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 08:59:52.911322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4f7c1-5056-4599-8c34-98bbec8a7a22 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:52.911917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4f7c1-5056-4599-8c34-98bbec8a7a22 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:52.911917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4f7c1-5056-4599-8c34-98bbec8a7a22 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:52.913239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 08:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 08:59:52.919528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444a110-bad8-4743-89b9-c6d4b680d0bf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:52.919919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444a110-bad8-4743-89b9-c6d4b680d0bf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:52.920080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444a110-bad8-4743-89b9-c6d4b680d0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:52.983977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e444a110-bad8-4743-89b9-c6d4b680d0bf None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 08:59:52.985181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e444a110-bad8-4743-89b9-c6d4b680d0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:53.033022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:53.080837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 08:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 08:59:53.117696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4f7c1-5056-4599-8c34-98bbec8a7a22 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 08:59:53.119260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a8a4f7c1-5056-4599-8c34-98bbec8a7a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:53.220854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4f7c1-5056-4599-8c34-98bbec8a7a22 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 08:59:53.267628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 08:59:53.293914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604f338-aadc-4e8e-ad86-064b9f0b860c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 08:59:53.294516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604f338-aadc-4e8e-ad86-064b9f0b860c service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 08:59:53.294799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604f338-aadc-4e8e-ad86-064b9f0b860c service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 08:59:53.295077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604f338-aadc-4e8e-ad86-064b9f0b860c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 08:59:53.296704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4604f338-aadc-4e8e-ad86-064b9f0b860c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 08:59:53.406067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604f338-aadc-4e8e-ad86-064b9f0b860c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 08:59:53.407630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604f338-aadc-4e8e-ad86-064b9f0b860c service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 08:59:53.425771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 742/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 08:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:00:03.079043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=22 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:03.423487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=66 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:13.150446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dd566-85ad-479a-8d52-299366f1c370 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:13.151027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dd566-85ad-479a-8d52-299366f1c370 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:13.151027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dd566-85ad-479a-8d52-299366f1c370 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:13.152359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 729/1471] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 09:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:00:13.157833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd4cf1-05d7-4eaf-bb35-a8afa87656ed None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:13.158281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd4cf1-05d7-4eaf-bb35-a8afa87656ed None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:13.158412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd4cf1-05d7-4eaf-bb35-a8afa87656ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:13.224315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f7cd4cf1-05d7-4eaf-bb35-a8afa87656ed None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:00:13.225503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f7cd4cf1-05d7-4eaf-bb35-a8afa87656ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:13.321657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 743/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 09:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:00:13.403952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d10b2-dd17-4753-8c9e-afda2a3de057 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:13.404184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d10b2-dd17-4753-8c9e-afda2a3de057 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:13.404464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d10b2-dd17-4753-8c9e-afda2a3de057 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:13.406076 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 730/1473] 10.222.0.40 () {54 vars in 962 bytes} [Wed Jul 13 09:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:00:13.410749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97dbf0-e838-41bb-b275-535116c00dd1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:13.411040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97dbf0-e838-41bb-b275-535116c00dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:13.411322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97dbf0-e838-41bb-b275-535116c00dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:13.484687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ca97dbf0-e838-41bb-b275-535116c00dd1 None None] MFA Rules not processed for user `cf67fa55f4d64b7ebeeb1490453661e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:00:13.485891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ca97dbf0-e838-41bb-b275-535116c00dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:13.506792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:13.563584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 744/1474] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Jul 13 09:00:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:00:13.571537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d10b2-dd17-4753-8c9e-afda2a3de057 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:00:13.574175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cd7d10b2-dd17-4753-8c9e-afda2a3de057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:13.677049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d10b2-dd17-4753-8c9e-afda2a3de057 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:00:13.750385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9993a-a359-4cac-83da-a6803cbe70a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf67fa55f4d64b7ebeeb1490453661e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:00:13.751023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9993a-a359-4cac-83da-a6803cbe70a1 service glance] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:13.751299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9993a-a359-4cac-83da-a6803cbe70a1 service glance] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:13.751568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9993a-a359-4cac-83da-a6803cbe70a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:13.753179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7cb9993a-a359-4cac-83da-a6803cbe70a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:13.806084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:13.859335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9993a-a359-4cac-83da-a6803cbe70a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:00:13.860749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9993a-a359-4cac-83da-a6803cbe70a1 service glance] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:00:13.879333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 731/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:00:16.011389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70299a52-72cb-400c-ab92-a38b9695fb5f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:16.011961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70299a52-72cb-400c-ab92-a38b9695fb5f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:16.011961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70299a52-72cb-400c-ab92-a38b9695fb5f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:16.013355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 745/1476] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 09:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:00:16.018771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046516c-6c1b-4d47-9359-59bd66f1daf5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:16.019194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046516c-6c1b-4d47-9359-59bd66f1daf5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:16.019301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046516c-6c1b-4d47-9359-59bd66f1daf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:16.083974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9046516c-6c1b-4d47-9359-59bd66f1daf5 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:00:16.085190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9046516c-6c1b-4d47-9359-59bd66f1daf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:16.186789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 732/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 09:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:00:16.210831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70299a52-72cb-400c-ab92-a38b9695fb5f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:00:16.212356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-70299a52-72cb-400c-ab92-a38b9695fb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:16.338306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70299a52-72cb-400c-ab92-a38b9695fb5f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:00:16.387680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:16.432790 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f339ff-0d92-4184-b78d-e1d0c07315dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:00:16.433446 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f339ff-0d92-4184-b78d-e1d0c07315dc admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:16.433723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f339ff-0d92-4184-b78d-e1d0c07315dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:16.433993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f339ff-0d92-4184-b78d-e1d0c07315dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:16.434743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f339ff-0d92-4184-b78d-e1d0c07315dc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:00:16.436538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-32f339ff-0d92-4184-b78d-e1d0c07315dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:00:16.436538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 13 09:00:16.436538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: Jul 13 09:00:16.437666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f339ff-0d92-4184-b78d-e1d0c07315dc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:00:16.457259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 746/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 13 09:00:16 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:00:18.561937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a8de3-f40c-45b1-823b-9a01765c94f8 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:18.562600 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a8de3-f40c-45b1-823b-9a01765c94f8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:18.562600 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a8de3-f40c-45b1-823b-9a01765c94f8 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:18.563905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 733/1479] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 09:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:00:18.569204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4b152-65c9-404f-9625-ba9fdf0a56a5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:18.575061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4b152-65c9-404f-9625-ba9fdf0a56a5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:18.575061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4b152-65c9-404f-9625-ba9fdf0a56a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:18.640054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-84f4b152-65c9-404f-9625-ba9fdf0a56a5 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:00:18.641278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-84f4b152-65c9-404f-9625-ba9fdf0a56a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:18.748186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 747/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 09:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:00:18.796317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a8de3-f40c-45b1-823b-9a01765c94f8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:00:18.797827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dc9a8de3-f40c-45b1-823b-9a01765c94f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:18.808270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:18.957669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a8de3-f40c-45b1-823b-9a01765c94f8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:00:19.039011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:19.071437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48003c9a-d1ab-45be-8b13-48bb4c39cfea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:00:19.072083 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48003c9a-d1ab-45be-8b13-48bb4c39cfea service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:19.072364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48003c9a-d1ab-45be-8b13-48bb4c39cfea service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:19.072639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48003c9a-d1ab-45be-8b13-48bb4c39cfea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:19.074330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-48003c9a-d1ab-45be-8b13-48bb4c39cfea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:19.164821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48003c9a-d1ab-45be-8b13-48bb4c39cfea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:00:19.172586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48003c9a-d1ab-45be-8b13-48bb4c39cfea service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:00:19.184889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 734/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:00:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:00:21.389353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015750e5-4ded-41f9-ac91-4ea3e0e6b516 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:21.389871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015750e5-4ded-41f9-ac91-4ea3e0e6b516 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:21.389980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015750e5-4ded-41f9-ac91-4ea3e0e6b516 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:21.391308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 748/1482] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 09:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:00:21.396419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67566bcb-bb57-4d79-912c-15de1a7ff6cd None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:21.396840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67566bcb-bb57-4d79-912c-15de1a7ff6cd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:21.396972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67566bcb-bb57-4d79-912c-15de1a7ff6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:21.461492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-67566bcb-bb57-4d79-912c-15de1a7ff6cd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:00:21.462715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-67566bcb-bb57-4d79-912c-15de1a7ff6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:21.509381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:21.559833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 735/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 09:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:00:21.595779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015750e5-4ded-41f9-ac91-4ea3e0e6b516 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:00:21.597298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-015750e5-4ded-41f9-ac91-4ea3e0e6b516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:21.621840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:21.700461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015750e5-4ded-41f9-ac91-4ea3e0e6b516 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:00:21.772377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcb2e4-1374-43fe-a9c8-8241b56d7f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:00:21.773717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb2e4-1374-43fe-a9c8-8241b56d7f19 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:21.774000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb2e4-1374-43fe-a9c8-8241b56d7f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:21.774308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb2e4-1374-43fe-a9c8-8241b56d7f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:21.775943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1adcb2e4-1374-43fe-a9c8-8241b56d7f19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:21.849050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:21.883430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcb2e4-1374-43fe-a9c8-8241b56d7f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:00:21.885346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcb2e4-1374-43fe-a9c8-8241b56d7f19 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:00:21.904149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 749/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:00:24.008918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8df1ea-056b-40ed-b7b4-1c6e7af72f5d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:24.009490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8df1ea-056b-40ed-b7b4-1c6e7af72f5d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:24.009490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8df1ea-056b-40ed-b7b4-1c6e7af72f5d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:24.010918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 736/1485] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 09:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:00:24.016279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a3839-cc03-4aee-8b58-b3a2006f64d1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:24.016717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a3839-cc03-4aee-8b58-b3a2006f64d1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:24.016843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a3839-cc03-4aee-8b58-b3a2006f64d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:24.082235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0a2a3839-cc03-4aee-8b58-b3a2006f64d1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:00:24.083428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0a2a3839-cc03-4aee-8b58-b3a2006f64d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:24.178519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 750/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 09:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:00:24.215879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8df1ea-056b-40ed-b7b4-1c6e7af72f5d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:00:24.217405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9a8df1ea-056b-40ed-b7b4-1c6e7af72f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:24.322513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8df1ea-056b-40ed-b7b4-1c6e7af72f5d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:00:24.400223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:24.401956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32f53f-0d8b-4cfb-b9ca-0d9a1d552906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:00:24.402629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f53f-0d8b-4cfb-b9ca-0d9a1d552906 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:24.402911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f53f-0d8b-4cfb-b9ca-0d9a1d552906 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:24.403184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f53f-0d8b-4cfb-b9ca-0d9a1d552906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:24.404808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9e32f53f-0d8b-4cfb-b9ca-0d9a1d552906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:24.514472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32f53f-0d8b-4cfb-b9ca-0d9a1d552906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:00:24.515859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32f53f-0d8b-4cfb-b9ca-0d9a1d552906 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:00:24.534974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 737/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:00:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:00:26.655389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d212-83c9-445c-b87c-43f87a20cd9f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:26.655931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d212-83c9-445c-b87c-43f87a20cd9f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:26.655931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d212-83c9-445c-b87c-43f87a20cd9f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:26.657284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 751/1488] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 09:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:00:26.662393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08514d69-37f9-4767-b65f-9fec02fdd71f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:26.662818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08514d69-37f9-4767-b65f-9fec02fdd71f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:26.663098 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08514d69-37f9-4767-b65f-9fec02fdd71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:26.726753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-08514d69-37f9-4767-b65f-9fec02fdd71f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:00:26.727958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-08514d69-37f9-4767-b65f-9fec02fdd71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:26.779720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:26.823965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 738/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 09:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:00:26.863781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25d212-83c9-445c-b87c-43f87a20cd9f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:00:26.865326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fc25d212-83c9-445c-b87c-43f87a20cd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:26.901581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:26.968492 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25d212-83c9-445c-b87c-43f87a20cd9f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:00:27.043341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d09134-7721-4f32-abea-59a711f184a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:00:27.043974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09134-7721-4f32-abea-59a711f184a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:27.044252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09134-7721-4f32-abea-59a711f184a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:27.044519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09134-7721-4f32-abea-59a711f184a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:27.046178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-90d09134-7721-4f32-abea-59a711f184a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:27.125931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:27.155147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d09134-7721-4f32-abea-59a711f184a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:00:27.156601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d09134-7721-4f32-abea-59a711f184a9 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:00:27.175829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 752/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:00:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:00:29.555800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0524c-325a-47de-a9d4-cc25cbede4e6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:29.556339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0524c-325a-47de-a9d4-cc25cbede4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:29.556444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0524c-325a-47de-a9d4-cc25cbede4e6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:29.557724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 739/1491] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 13 09:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 09:00:29.563024 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aab353-9622-40bb-abe5-3cb0b4fd2e4d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:29.563467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aab353-9622-40bb-abe5-3cb0b4fd2e4d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:29.563578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aab353-9622-40bb-abe5-3cb0b4fd2e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:29.628787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-82aab353-9622-40bb-abe5-3cb0b4fd2e4d None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:00:29.629973 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-82aab353-9622-40bb-abe5-3cb0b4fd2e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:29.728091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 753/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 13 09:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:00:29.767059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0524c-325a-47de-a9d4-cc25cbede4e6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:00:29.768639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-37d0524c-325a-47de-a9d4-cc25cbede4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:29.874008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0524c-325a-47de-a9d4-cc25cbede4e6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:00:29.946229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847dc94-c5bb-4340-9f15-516fa7dc5871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:00:29.946792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847dc94-c5bb-4340-9f15-516fa7dc5871 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:00:29.947070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847dc94-c5bb-4340-9f15-516fa7dc5871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:00:29.947342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847dc94-c5bb-4340-9f15-516fa7dc5871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:00:29.948949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3847dc94-c5bb-4340-9f15-516fa7dc5871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:00:29.957782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:30.058400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847dc94-c5bb-4340-9f15-516fa7dc5871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:00:30.059829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847dc94-c5bb-4340-9f15-516fa7dc5871 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:00:30.077358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 740/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:00:39.726372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=82 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:00:40.075171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=50 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:02:25.385614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d700fc0-6337-4952-8870-841e527db8fd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:25.386173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d700fc0-6337-4952-8870-841e527db8fd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:25.386270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d700fc0-6337-4952-8870-841e527db8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:25.469270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8d700fc0-6337-4952-8870-841e527db8fd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:02:25.470511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8d700fc0-6337-4952-8870-841e527db8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:02:25.573981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 754/1494] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:02:25.582330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2ee76-f3f9-4a3d-b892-4f1d58a6c5e7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:25.582756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2ee76-f3f9-4a3d-b892-4f1d58a6c5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:25.582943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2ee76-f3f9-4a3d-b892-4f1d58a6c5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:25.601882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-47c2ee76-f3f9-4a3d-b892-4f1d58a6c5e7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:02:25.603033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-47c2ee76-f3f9-4a3d-b892-4f1d58a6c5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:02:25.611272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 741/1495] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:02:25.616528 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d700fc0-6337-4952-8870-841e527db8fd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:02:25.626142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d700fc0-6337-4952-8870-841e527db8fd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:02:25.630777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6e669-fe26-49cc-ae2d-29ec8db4c714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:02:25.631419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e669-fe26-49cc-ae2d-29ec8db4c714 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:25.631699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e669-fe26-49cc-ae2d-29ec8db4c714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:25.631971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6e669-fe26-49cc-ae2d-29ec8db4c714 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:25.632692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6e669-fe26-49cc-ae2d-29ec8db4c714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:02:25.633016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6e669-fe26-49cc-ae2d-29ec8db4c714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:02:25.634020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-8cf6e669-fe26-49cc-ae2d-29ec8db4c714 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:02:25.634020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 13 09:02:25.634020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: Jul 13 09:02:25.634941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6e669-fe26-49cc-ae2d-29ec8db4c714 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:02:25.646620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 755/1496] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:02:25.654704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2ee76-f3f9-4a3d-b892-4f1d58a6c5e7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:02:25.664400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2ee76-f3f9-4a3d-b892-4f1d58a6c5e7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:02:25.668985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63878cd2-e412-4076-b5af-076a8ab9f3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:02:25.669628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63878cd2-e412-4076-b5af-076a8ab9f3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:25.669910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63878cd2-e412-4076-b5af-076a8ab9f3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:25.670222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63878cd2-e412-4076-b5af-076a8ab9f3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:25.670918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63878cd2-e412-4076-b5af-076a8ab9f3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:02:25.672658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-63878cd2-e412-4076-b5af-076a8ab9f3e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:02:25.672658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 13 09:02:25.672658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: Jul 13 09:02:25.673860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63878cd2-e412-4076-b5af-076a8ab9f3e5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:02:25.795003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 742/1497] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:02:25 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:02:25.799947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6e669-fe26-49cc-ae2d-29ec8db4c714 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:02:25.809986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6e669-fe26-49cc-ae2d-29ec8db4c714 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:02:25.859703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:02:25.866484 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224c101-d259-4900-9558-0f84c3248cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:02:25.867149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224c101-d259-4900-9558-0f84c3248cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:25.867430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224c101-d259-4900-9558-0f84c3248cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:25.867703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224c101-d259-4900-9558-0f84c3248cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:25.868393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224c101-d259-4900-9558-0f84c3248cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:02:25.869411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-3224c101-d259-4900-9558-0f84c3248cd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:02:25.869411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 13 09:02:25.869411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: Jul 13 09:02:25.870643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224c101-d259-4900-9558-0f84c3248cd7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:02:25.993696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 756/1498] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:02:25 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:02:25.997831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63878cd2-e412-4076-b5af-076a8ab9f3e5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:02:26.008486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63878cd2-e412-4076-b5af-076a8ab9f3e5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:02:26.013120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31925be-b8e6-4d77-8645-f80a3082289d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:02:26.013783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31925be-b8e6-4d77-8645-f80a3082289d admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:26.014158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31925be-b8e6-4d77-8645-f80a3082289d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:26.014436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31925be-b8e6-4d77-8645-f80a3082289d admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:26.015192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31925be-b8e6-4d77-8645-f80a3082289d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:02:26.016189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-c31925be-b8e6-4d77-8645-f80a3082289d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:02:26.016189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 13 09:02:26.016189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: Jul 13 09:02:26.017070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31925be-b8e6-4d77-8645-f80a3082289d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:02:26.030832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 743/1499] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:02:26.035793 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224c101-d259-4900-9558-0f84c3248cd7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:02:26.045673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224c101-d259-4900-9558-0f84c3248cd7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:02:26.050483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63906432-2b90-4fe2-a77b-5ce8aee88ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:02:26.051129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63906432-2b90-4fe2-a77b-5ce8aee88ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:26.051413 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63906432-2b90-4fe2-a77b-5ce8aee88ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:26.051692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63906432-2b90-4fe2-a77b-5ce8aee88ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:26.052410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63906432-2b90-4fe2-a77b-5ce8aee88ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:02:26.053415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-63906432-2b90-4fe2-a77b-5ce8aee88ef6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:02:26.053415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 13 09:02:26.053415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: Jul 13 09:02:26.054319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63906432-2b90-4fe2-a77b-5ce8aee88ef6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:02:26.067918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 757/1500] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:02:26.071914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31925be-b8e6-4d77-8645-f80a3082289d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:02:26.082364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31925be-b8e6-4d77-8645-f80a3082289d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:02:26.086855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c5be4-6ec7-4977-9cf4-768f90166c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:02:26.087720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c5be4-6ec7-4977-9cf4-768f90166c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:26.088001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c5be4-6ec7-4977-9cf4-768f90166c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:26.088276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c5be4-6ec7-4977-9cf4-768f90166c8d admin admin] PATH_INFO: `/v3/projects/69af276b78f24ac4a34afe7bcfb15a83/users/d9eaae6908de44aa8dfc431b5e399add/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:26.103440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c5be4-6ec7-4977-9cf4-768f90166c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=69af276b78f24ac4a34afe7bcfb15a83, user_id=d9eaae6908de44aa8dfc431b5e399add, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:02:26.104452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-771c5be4-6ec7-4977-9cf4-768f90166c8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:02:26.104452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 13 09:02:26.104452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: Jul 13 09:02:26.106396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c5be4-6ec7-4977-9cf4-768f90166c8d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:02:26.170847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 744/1501] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:02:26 2022] PUT /identity/v3/projects/69af276b78f24ac4a34afe7bcfb15a83/users/d9eaae6908de44aa8dfc431b5e399add/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:02:26.187039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8b86f-3b83-4399-87f3-4b496d14b48c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:26.187333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8b86f-3b83-4399-87f3-4b496d14b48c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:26.187612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8b86f-3b83-4399-87f3-4b496d14b48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:26.214440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b1d8b86f-3b83-4399-87f3-4b496d14b48c None None] MFA Rules not processed for user `d9eaae6908de44aa8dfc431b5e399add`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:02:26.215638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b1d8b86f-3b83-4399-87f3-4b496d14b48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:02:26.279109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 758/1502] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:02:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:02:26.292154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132d27c-07ad-49cc-8c43-642716f4297b None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:26.292445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132d27c-07ad-49cc-8c43-642716f4297b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:26.292722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132d27c-07ad-49cc-8c43-642716f4297b None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:26.294087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 745/1503] 10.222.0.40 () {54 vars in 963 bytes} [Wed Jul 13 09:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:02:26.298704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d5bc7-bb5f-43e4-9191-f43754f0f3d5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:26.299062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d5bc7-bb5f-43e4-9191-f43754f0f3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:26.299549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d5bc7-bb5f-43e4-9191-f43754f0f3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:26.358515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ff8d5bc7-bb5f-43e4-9191-f43754f0f3d5 None None] MFA Rules not processed for user `b3f12750f91645748b37e2b159a9bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:02:26.359727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ff8d5bc7-bb5f-43e4-9191-f43754f0f3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:02:26.445471 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 759/1504] 10.222.0.40 () {58 vars in 1060 bytes} [Wed Jul 13 09:02:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:02:26.453153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132d27c-07ad-49cc-8c43-642716f4297b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:02:26.464189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132d27c-07ad-49cc-8c43-642716f4297b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:02:26.467675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380b6f8-ac9a-4105-b56c-3d0db40edf9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:02:26.468299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b6f8-ac9a-4105-b56c-3d0db40edf9b service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:26.468575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b6f8-ac9a-4105-b56c-3d0db40edf9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:26.468848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b6f8-ac9a-4105-b56c-3d0db40edf9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:26.477161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380b6f8-ac9a-4105-b56c-3d0db40edf9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:02:26.478611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380b6f8-ac9a-4105-b56c-3d0db40edf9b service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:02:26.487232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 746/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:02:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:02:26.532878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dd167-3bde-49e2-8604-962de6ff79e6 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:26.533696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dd167-3bde-49e2-8604-962de6ff79e6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:26.533696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dd167-3bde-49e2-8604-962de6ff79e6 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:26.535835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 760/1506] 10.222.0.40 () {52 vars in 829 bytes} [Wed Jul 13 09:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:02:26.578342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380b6f8-ac9a-4105-b56c-3d0db40edf9b service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:02:26.641330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380b6f8-ac9a-4105-b56c-3d0db40edf9b service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:02:26.646089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:02:26.647006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:26.647288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:26.647560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] PATH_INFO: `/v3/users/d9eaae6908de44aa8dfc431b5e399add` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:26.648769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9eaae6908de44aa8dfc431b5e399add)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:02:26.649775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:02:26.649775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 13 09:02:26.649775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: Jul 13 09:02:26.651323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:02:26.923460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9eaae6908de44aa8dfc431b5e399add'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:02:26.936270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9eaae6908de44aa8dfc431b5e399add'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:02:26.954551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9eaae6908de44aa8dfc431b5e399add'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:02:26.958706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:02:27.025221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9eaae6908de44aa8dfc431b5e399add'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:02:27.025836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9eaae6908de44aa8dfc431b5e399add'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:02:27.032232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 747/1507] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:02:26 2022] DELETE /identity/v3/users/d9eaae6908de44aa8dfc431b5e399add => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:02:27.037353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc8eb5-1721-48f5-966b-66e363088270 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:27.037770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc8eb5-1721-48f5-966b-66e363088270 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:27.038077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc8eb5-1721-48f5-966b-66e363088270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:27.055950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e5bc8eb5-1721-48f5-966b-66e363088270 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:02:27.057130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e5bc8eb5-1721-48f5-966b-66e363088270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:02:27.066395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:02:27.116182 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 761/1508] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:02:27.125585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:02:27.127012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:02:27.203355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d81e-f708-452c-bfaa-d9a41cda5beb admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:02:27.207111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6296589-4454-4e7f-907b-5c04383231eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:02:27.207761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6296589-4454-4e7f-907b-5c04383231eb service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:27.208040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6296589-4454-4e7f-907b-5c04383231eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:27.208316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6296589-4454-4e7f-907b-5c04383231eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:27.217202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6296589-4454-4e7f-907b-5c04383231eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:02:27.218667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6296589-4454-4e7f-907b-5c04383231eb service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:02:27.228461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 748/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:02:27.689780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc8eb5-1721-48f5-966b-66e363088270 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:02:27.691958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e5bc8eb5-1721-48f5-966b-66e363088270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:02:27.707461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc8eb5-1721-48f5-966b-66e363088270 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:02:27.710354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:02:27.711208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:02:27.711488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:02:27.711760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] PATH_INFO: `/v3/projects/69af276b78f24ac4a34afe7bcfb15a83` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:02:27.712948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=69af276b78f24ac4a34afe7bcfb15a83)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:02:27.713974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:02:27.713974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 13 09:02:27.713974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: Jul 13 09:02:27.715189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:02:27.842602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69af276b78f24ac4a34afe7bcfb15a83', 'request_id': 'req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:02:27.848604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69af276b78f24ac4a34afe7bcfb15a83', 'request_id': 'req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:02:27.861740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69af276b78f24ac4a34afe7bcfb15a83', 'request_id': 'req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:02:27.947306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] The token cache is being invalidate because project 69af276b78f24ac4a34afe7bcfb15a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:02:27.947589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:02:27.948305 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f1e1c9f2-fa38-42c7-ad7c-f625457519bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:02:27.949858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 762/1510] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:02:27 2022] DELETE /identity/v3/projects/69af276b78f24ac4a34afe7bcfb15a83 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:02:37.224361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:02:37.944174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:04:03.534246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b138253-80d1-4489-95da-c847647d1b9e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:03.534879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b138253-80d1-4489-95da-c847647d1b9e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:03.534879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b138253-80d1-4489-95da-c847647d1b9e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:03.536361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 09:04:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:04:03.541489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4ba79-123b-43a7-9c88-3eb3d06fcb76 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:03.541961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4ba79-123b-43a7-9c88-3eb3d06fcb76 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:03.542098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4ba79-123b-43a7-9c88-3eb3d06fcb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:03.560541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1ee4ba79-123b-43a7-9c88-3eb3d06fcb76 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:04:03.561754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1ee4ba79-123b-43a7-9c88-3eb3d06fcb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:03.620606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 09:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:04:03.775425 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b138253-80d1-4489-95da-c847647d1b9e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:04:03.779089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4b138253-80d1-4489-95da-c847647d1b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:03.879720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b138253-80d1-4489-95da-c847647d1b9e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:04:03.885070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170809b-ee4d-45d8-8953-971e8857c84e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:04:03.885709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170809b-ee4d-45d8-8953-971e8857c84e service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:03.885988 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170809b-ee4d-45d8-8953-971e8857c84e service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:03.886292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170809b-ee4d-45d8-8953-971e8857c84e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:03.895320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170809b-ee4d-45d8-8953-971e8857c84e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:04:03.896917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170809b-ee4d-45d8-8953-971e8857c84e service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:04:03.906578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 750/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:04:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:04:04.739530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11132db9-ac48-43d5-8daf-65a7b315b414 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:04.740021 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11132db9-ac48-43d5-8daf-65a7b315b414 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:04.740124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11132db9-ac48-43d5-8daf-65a7b315b414 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:04.741495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 09:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:04:04.746453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52859ab-06fe-40fb-ac74-9779febe3b8c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:04.747030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52859ab-06fe-40fb-ac74-9779febe3b8c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:04.747030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52859ab-06fe-40fb-ac74-9779febe3b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:04.765939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e52859ab-06fe-40fb-ac74-9779febe3b8c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:04:04.767148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e52859ab-06fe-40fb-ac74-9779febe3b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:04.775414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 09:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:04:04.926237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11132db9-ac48-43d5-8daf-65a7b315b414 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:04:04.940842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11132db9-ac48-43d5-8daf-65a7b315b414 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:04:04.946251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598dbb5-24c6-43c8-95dc-9efcb188d6d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:04:04.946882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598dbb5-24c6-43c8-95dc-9efcb188d6d7 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:04.947163 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598dbb5-24c6-43c8-95dc-9efcb188d6d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:04.947437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598dbb5-24c6-43c8-95dc-9efcb188d6d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:04.955569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598dbb5-24c6-43c8-95dc-9efcb188d6d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:04:04.957157 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598dbb5-24c6-43c8-95dc-9efcb188d6d7 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:04:04.966416 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 765/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:04:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:04:05.913974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50855d-8d86-4e93-a2ef-b4e091165f41 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:05.914553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50855d-8d86-4e93-a2ef-b4e091165f41 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:05.914553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50855d-8d86-4e93-a2ef-b4e091165f41 None None] PATH_INFO: `/v3` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:05.915895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 752/1517] 10.222.0.30 () {54 vars in 925 bytes} [Wed Jul 13 09:04:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:04:05.921774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06335e70-c724-4966-ac55-dde4a205529a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:05.922211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06335e70-c724-4966-ac55-dde4a205529a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:05.922383 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06335e70-c724-4966-ac55-dde4a205529a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:05.969771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-06335e70-c724-4966-ac55-dde4a205529a None None] MFA Rules not processed for user `bca7c1164d3e4833a10ffc37cbc6f092`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:04:05.971091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-06335e70-c724-4966-ac55-dde4a205529a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:06.041327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 766/1518] 10.222.0.30 () {58 vars in 1015 bytes} [Wed Jul 13 09:04:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:04:10.116428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b7de3-a02c-45ce-9ba8-af7416618a29 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:10.116972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b7de3-a02c-45ce-9ba8-af7416618a29 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:10.116972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b7de3-a02c-45ce-9ba8-af7416618a29 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:10.118356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 753/1519] 10.222.0.40 () {54 vars in 960 bytes} [Wed Jul 13 09:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:04:10.123420 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3963ddb-70d3-4929-9ac1-d6a2bad446cc None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:10.123851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3963ddb-70d3-4929-9ac1-d6a2bad446cc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:10.123964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3963ddb-70d3-4929-9ac1-d6a2bad446cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:10.144153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d3963ddb-70d3-4929-9ac1-d6a2bad446cc None None] MFA Rules not processed for user `bca7c1164d3e4833a10ffc37cbc6f092`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:04:10.145339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d3963ddb-70d3-4929-9ac1-d6a2bad446cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:10.154065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 767/1520] 10.222.0.40 () {58 vars in 1057 bytes} [Wed Jul 13 09:04:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:04:10.161548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b7de3-a02c-45ce-9ba8-af7416618a29 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:04:10.175123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b7de3-a02c-45ce-9ba8-af7416618a29 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:04:10.180077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723f8f8-fbee-4a6f-9b87-9a20e606f354 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:04:10.180717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f8f8-fbee-4a6f-9b87-9a20e606f354 service placement] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:10.180979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f8f8-fbee-4a6f-9b87-9a20e606f354 service placement] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:10.181253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f8f8-fbee-4a6f-9b87-9a20e606f354 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:10.189263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723f8f8-fbee-4a6f-9b87-9a20e606f354 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:04:10.191604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723f8f8-fbee-4a6f-9b87-9a20e606f354 service placement] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:04:10.201398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 754/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:04:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:04:16.209231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da3b26-7c66-46bd-86ef-de52e4603745 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:16.209742 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da3b26-7c66-46bd-86ef-de52e4603745 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:16.209837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da3b26-7c66-46bd-86ef-de52e4603745 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:16.211170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 09:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:04:16.216074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211479f-d822-4a55-9ad8-c3f9338f867f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:16.218749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211479f-d822-4a55-9ad8-c3f9338f867f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:16.218749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211479f-d822-4a55-9ad8-c3f9338f867f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:16.235072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f211479f-d822-4a55-9ad8-c3f9338f867f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:04:16.236253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f211479f-d822-4a55-9ad8-c3f9338f867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:16.244467 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 09:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:04:16.397912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da3b26-7c66-46bd-86ef-de52e4603745 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:04:16.412189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da3b26-7c66-46bd-86ef-de52e4603745 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:04:16.417585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0ff92-8287-4a56-991b-5c27b8b3a429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:04:16.418246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0ff92-8287-4a56-991b-5c27b8b3a429 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:16.418526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0ff92-8287-4a56-991b-5c27b8b3a429 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:16.418863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0ff92-8287-4a56-991b-5c27b8b3a429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:16.430183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0ff92-8287-4a56-991b-5c27b8b3a429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:04:16.430183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0ff92-8287-4a56-991b-5c27b8b3a429 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:04:16.438149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 769/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:04:17.255790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecae67-8419-47c2-be6c-327a48164f2b None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:17.256366 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecae67-8419-47c2-be6c-327a48164f2b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:17.256366 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecae67-8419-47c2-be6c-327a48164f2b None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:17.257690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 09:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 09:04:17.262473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a10d9-9a44-4891-8b07-8e6b4f1fc801 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:17.262870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a10d9-9a44-4891-8b07-8e6b4f1fc801 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:17.263051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a10d9-9a44-4891-8b07-8e6b4f1fc801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:17.400280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-180a10d9-9a44-4891-8b07-8e6b4f1fc801 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:04:17.401499 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-180a10d9-9a44-4891-8b07-8e6b4f1fc801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:17.409415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 09:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:04:17.560223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ecae67-8419-47c2-be6c-327a48164f2b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:04:17.574696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ecae67-8419-47c2-be6c-327a48164f2b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:04:17.580031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784ec6a-74bb-42c6-b76a-610d835d04ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:04:17.580687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784ec6a-74bb-42c6-b76a-610d835d04ec service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:17.580962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784ec6a-74bb-42c6-b76a-610d835d04ec service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:17.581230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784ec6a-74bb-42c6-b76a-610d835d04ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:17.590088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784ec6a-74bb-42c6-b76a-610d835d04ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:04:17.591640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784ec6a-74bb-42c6-b76a-610d835d04ec service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:04:17.601591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 757/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:04:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:04:21.571194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d26630-b153-43ea-ad53-de66ed34f590 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:21.571782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d26630-b153-43ea-ad53-de66ed34f590 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:21.571782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d26630-b153-43ea-ad53-de66ed34f590 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:21.573090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 09:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:04:21.578182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f91890-3498-4654-ae9b-58865b0d4edd None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:21.578622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f91890-3498-4654-ae9b-58865b0d4edd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:21.578765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f91890-3498-4654-ae9b-58865b0d4edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:21.597362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f5f91890-3498-4654-ae9b-58865b0d4edd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:04:21.598529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f5f91890-3498-4654-ae9b-58865b0d4edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:21.606226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 09:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:04:21.678211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d26630-b153-43ea-ad53-de66ed34f590 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:04:21.679912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d1d26630-b153-43ea-ad53-de66ed34f590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:21.707685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:04:21.737191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d26630-b153-43ea-ad53-de66ed34f590 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:04:21.741078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70f913-c590-4a53-93a9-d8ec7d2e84ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:04:21.741770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70f913-c590-4a53-93a9-d8ec7d2e84ce service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:21.742190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70f913-c590-4a53-93a9-d8ec7d2e84ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:21.742749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70f913-c590-4a53-93a9-d8ec7d2e84ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:21.751490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70f913-c590-4a53-93a9-d8ec7d2e84ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:04:21.752913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70f913-c590-4a53-93a9-d8ec7d2e84ce service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:04:21.762943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 772/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:04:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:04:23.827311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a78fff-02af-4d1e-b775-c2237875bc4e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:23.827874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a78fff-02af-4d1e-b775-c2237875bc4e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:23.827874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a78fff-02af-4d1e-b775-c2237875bc4e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:23.829230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 09:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:04:23.835064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558da42-a6a5-4f2e-a5ef-f586e35c961a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:23.835522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558da42-a6a5-4f2e-a5ef-f586e35c961a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:23.835623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558da42-a6a5-4f2e-a5ef-f586e35c961a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:23.853796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a558da42-a6a5-4f2e-a5ef-f586e35c961a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:04:23.855025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a558da42-a6a5-4f2e-a5ef-f586e35c961a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:23.862869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 09:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:04:23.899513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a78fff-02af-4d1e-b775-c2237875bc4e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:04:23.910277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a78fff-02af-4d1e-b775-c2237875bc4e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:04:23.914816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48feaa-ae7c-4ad9-9775-20180d873ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:04:23.916385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48feaa-ae7c-4ad9-9775-20180d873ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:23.916663 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48feaa-ae7c-4ad9-9775-20180d873ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:23.916933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48feaa-ae7c-4ad9-9775-20180d873ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:23.926548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48feaa-ae7c-4ad9-9775-20180d873ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:04:23.928581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48feaa-ae7c-4ad9-9775-20180d873ec3 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:04:23.937775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 760/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:04:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:04:27.387323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5498e5-4280-4b2a-a2bc-8abf23c29683 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:27.387867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5498e5-4280-4b2a-a2bc-8abf23c29683 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:27.387867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5498e5-4280-4b2a-a2bc-8abf23c29683 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:27.389244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 09:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:04:27.394370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16990a15-5733-4958-acc9-87211178d5d0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:27.394801 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16990a15-5733-4958-acc9-87211178d5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:27.394930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16990a15-5733-4958-acc9-87211178d5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:27.401596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=74 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:04:27.413401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-16990a15-5733-4958-acc9-87211178d5d0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:04:27.420175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-16990a15-5733-4958-acc9-87211178d5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:27.422629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 09:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:04:27.722284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5498e5-4280-4b2a-a2bc-8abf23c29683 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:04:27.724458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ec5498e5-4280-4b2a-a2bc-8abf23c29683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:27.781204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5498e5-4280-4b2a-a2bc-8abf23c29683 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:04:27.784752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c5706-fc09-42e3-9eb8-505ddf2b89e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:04:27.785389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c5706-fc09-42e3-9eb8-505ddf2b89e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:27.785673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c5706-fc09-42e3-9eb8-505ddf2b89e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:27.786505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c5706-fc09-42e3-9eb8-505ddf2b89e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:27.795293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c5706-fc09-42e3-9eb8-505ddf2b89e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:04:27.796714 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c5706-fc09-42e3-9eb8-505ddf2b89e5 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:04:27.806673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 775/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:04:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:04:29.871460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c03a6-12af-443e-bb52-0c29c27e9828 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:29.872036 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c03a6-12af-443e-bb52-0c29c27e9828 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:29.872036 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c03a6-12af-443e-bb52-0c29c27e9828 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:29.873371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 09:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:04:29.878396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbf8b2-04de-4575-9735-7dabb5e34392 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:29.878825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbf8b2-04de-4575-9735-7dabb5e34392 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:29.878956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbf8b2-04de-4575-9735-7dabb5e34392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:29.896292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8acbf8b2-04de-4575-9735-7dabb5e34392 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:04:29.897778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8acbf8b2-04de-4575-9735-7dabb5e34392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:29.905410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 09:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:04:30.208259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c03a6-12af-443e-bb52-0c29c27e9828 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:04:30.210200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e67c03a6-12af-443e-bb52-0c29c27e9828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:04:30.222878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c03a6-12af-443e-bb52-0c29c27e9828 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:04:30.227300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72490fed-a9c4-41cb-9023-dcfcfd45224b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:04:30.227925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72490fed-a9c4-41cb-9023-dcfcfd45224b service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:04:30.228201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72490fed-a9c4-41cb-9023-dcfcfd45224b service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:04:30.228485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72490fed-a9c4-41cb-9023-dcfcfd45224b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:04:30.237194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72490fed-a9c4-41cb-9023-dcfcfd45224b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:04:30.238623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72490fed-a9c4-41cb-9023-dcfcfd45224b service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:04:30.248131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 763/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:04:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:04:39.891530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=47 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:04:40.244130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=8 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:05:56.958761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a04a06-89c7-4590-a9ee-dd6892f6563f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:56.959414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a04a06-89c7-4590-a9ee-dd6892f6563f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:56.959414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a04a06-89c7-4590-a9ee-dd6892f6563f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:56.987074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b2a04a06-89c7-4590-a9ee-dd6892f6563f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:56.988827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b2a04a06-89c7-4590-a9ee-dd6892f6563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:56.997361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 777/1540] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:57.006292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f197de-0db4-4c73-a4c4-13cd5069d869 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.006860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f197de-0db4-4c73-a4c4-13cd5069d869 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.006860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f197de-0db4-4c73-a4c4-13cd5069d869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.031133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-86f197de-0db4-4c73-a4c4-13cd5069d869 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:57.032425 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-86f197de-0db4-4c73-a4c4-13cd5069d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:57.048834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 764/1541] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:57.053407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a04a06-89c7-4590-a9ee-dd6892f6563f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.067401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a04a06-89c7-4590-a9ee-dd6892f6563f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.078391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831a6a8-6740-4f36-bcfd-888bc49ee210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.079055 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831a6a8-6740-4f36-bcfd-888bc49ee210 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.079337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831a6a8-6740-4f36-bcfd-888bc49ee210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.079607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831a6a8-6740-4f36-bcfd-888bc49ee210 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.080331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831a6a8-6740-4f36-bcfd-888bc49ee210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:05:57.080634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831a6a8-6740-4f36-bcfd-888bc49ee210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.082795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831a6a8-6740-4f36-bcfd-888bc49ee210 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:57.104546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 778/1542] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:05:57.112444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f197de-0db4-4c73-a4c4-13cd5069d869 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.122918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f197de-0db4-4c73-a4c4-13cd5069d869 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.140467 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a54eb-f927-44fd-9300-48ad8b311ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.141136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a54eb-f927-44fd-9300-48ad8b311ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.141409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a54eb-f927-44fd-9300-48ad8b311ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.141693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a54eb-f927-44fd-9300-48ad8b311ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.142428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a54eb-f927-44fd-9300-48ad8b311ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.145121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a54eb-f927-44fd-9300-48ad8b311ee1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:57.243012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c612fd-bc25-4ba7-a07d-06cae047a8a4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.243200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c612fd-bc25-4ba7-a07d-06cae047a8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.243484 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c612fd-bc25-4ba7-a07d-06cae047a8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.258516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 765/1543] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:57.262657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a54eb-f927-44fd-9300-48ad8b311ee1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.271931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-81c612fd-bc25-4ba7-a07d-06cae047a8a4 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:57.273149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-81c612fd-bc25-4ba7-a07d-06cae047a8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:57.365990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a54eb-f927-44fd-9300-48ad8b311ee1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.374819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf028a-1479-4439-b681-b3671f85ee87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.375510 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf028a-1479-4439-b681-b3671f85ee87 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.375789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf028a-1479-4439-b681-b3671f85ee87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.376059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf028a-1479-4439-b681-b3671f85ee87 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.376755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf028a-1479-4439-b681-b3671f85ee87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.377772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-6acf028a-1479-4439-b681-b3671f85ee87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:05:57.377772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 13 09:05:57.377772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: Jul 13 09:05:57.387915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf028a-1479-4439-b681-b3671f85ee87 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:57.395861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 779/1544] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:57.412373 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abd42c-da1d-49b1-8355-29bc7cea1a5f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.417193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abd42c-da1d-49b1-8355-29bc7cea1a5f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.417193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abd42c-da1d-49b1-8355-29bc7cea1a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.446317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-34abd42c-da1d-49b1-8355-29bc7cea1a5f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:57.447522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-34abd42c-da1d-49b1-8355-29bc7cea1a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:57.468756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 780/1545] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:57.475203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abd42c-da1d-49b1-8355-29bc7cea1a5f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.477936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 766/1546] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:57.481679 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf028a-1479-4439-b681-b3671f85ee87 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.484831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abd42c-da1d-49b1-8355-29bc7cea1a5f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.495295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01c646-930e-490c-87bc-49f703779a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.495891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01c646-930e-490c-87bc-49f703779a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.496175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01c646-930e-490c-87bc-49f703779a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.496452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01c646-930e-490c-87bc-49f703779a8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.497183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01c646-930e-490c-87bc-49f703779a8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:05:57.497496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01c646-930e-490c-87bc-49f703779a8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.498116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf028a-1479-4439-b681-b3671f85ee87 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.499221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01c646-930e-490c-87bc-49f703779a8c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:57.506740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236367ff-b859-44d4-9a8b-a912aa7ef62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.514226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 781/1547] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:05:57.515446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236367ff-b859-44d4-9a8b-a912aa7ef62b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.515584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73279ef4-2602-4e6d-8939-183b5a4fd78b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.515872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73279ef4-2602-4e6d-8939-183b5a4fd78b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.516152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73279ef4-2602-4e6d-8939-183b5a4fd78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.516422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236367ff-b859-44d4-9a8b-a912aa7ef62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.516869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236367ff-b859-44d4-9a8b-a912aa7ef62b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.517880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236367ff-b859-44d4-9a8b-a912aa7ef62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.520904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236367ff-b859-44d4-9a8b-a912aa7ef62b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:57.540638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 767/1548] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:57.541824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236367ff-b859-44d4-9a8b-a912aa7ef62b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.563263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-73279ef4-2602-4e6d-8939-183b5a4fd78b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:57.564510 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-73279ef4-2602-4e6d-8939-183b5a4fd78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:57.569140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236367ff-b859-44d4-9a8b-a912aa7ef62b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.578943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb64b3-7f57-49c5-a50e-416914d89434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.579915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb64b3-7f57-49c5-a50e-416914d89434 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.580391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb64b3-7f57-49c5-a50e-416914d89434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.580829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb64b3-7f57-49c5-a50e-416914d89434 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.581711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb64b3-7f57-49c5-a50e-416914d89434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.584768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb64b3-7f57-49c5-a50e-416914d89434 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:57.595906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 782/1549] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:57.596784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73279ef4-2602-4e6d-8939-183b5a4fd78b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.611679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73279ef4-2602-4e6d-8939-183b5a4fd78b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.629314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deb699-fd63-4073-817b-48eac28ea392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.630096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deb699-fd63-4073-817b-48eac28ea392 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.630375 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deb699-fd63-4073-817b-48eac28ea392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.630649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deb699-fd63-4073-817b-48eac28ea392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.631374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deb699-fd63-4073-817b-48eac28ea392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.632997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deb699-fd63-4073-817b-48eac28ea392 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:57.645024 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 783/1550] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:57.646646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01ac25-e68e-44dc-b27e-2d9bb4783378 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.646937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01ac25-e68e-44dc-b27e-2d9bb4783378 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.647217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01ac25-e68e-44dc-b27e-2d9bb4783378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.675862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9a01ac25-e68e-44dc-b27e-2d9bb4783378 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:57.677215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9a01ac25-e68e-44dc-b27e-2d9bb4783378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:57.689512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 784/1551] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:57.690332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01ac25-e68e-44dc-b27e-2d9bb4783378 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.703811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01ac25-e68e-44dc-b27e-2d9bb4783378 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.715390 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 768/1552] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:57.716493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb64b3-7f57-49c5-a50e-416914d89434 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.721040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f870d96-c375-44ad-bae8-e7accbdd4e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.721040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f870d96-c375-44ad-bae8-e7accbdd4e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.721040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f870d96-c375-44ad-bae8-e7accbdd4e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.721040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f870d96-c375-44ad-bae8-e7accbdd4e2d admin admin] PATH_INFO: `/v3/projects/09b63eccabbc498cb96dd4bbcef0969a/users/1438ef91441f4493844a915ce12fcfa4/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.747564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f870d96-c375-44ad-bae8-e7accbdd4e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b63eccabbc498cb96dd4bbcef0969a, user_id=1438ef91441f4493844a915ce12fcfa4, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.748541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-6f870d96-c375-44ad-bae8-e7accbdd4e2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:05:57.748541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 13 09:05:57.748541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: Jul 13 09:05:57.750507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f870d96-c375-44ad-bae8-e7accbdd4e2d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:57.824829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb64b3-7f57-49c5-a50e-416914d89434 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.830180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a952277-56d5-4819-9ae1-a7730c925044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.837691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a952277-56d5-4819-9ae1-a7730c925044 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.837691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a952277-56d5-4819-9ae1-a7730c925044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.837691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a952277-56d5-4819-9ae1-a7730c925044 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.837691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a952277-56d5-4819-9ae1-a7730c925044 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:05:57.837691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a952277-56d5-4819-9ae1-a7730c925044 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.838221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-4a952277-56d5-4819-9ae1-a7730c925044 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:05:57.838221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 13 09:05:57.838221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: Jul 13 09:05:57.838221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a952277-56d5-4819-9ae1-a7730c925044 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:57.847258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 769/1553] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:05:57.848248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a952277-56d5-4819-9ae1-a7730c925044 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.858850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a952277-56d5-4819-9ae1-a7730c925044 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.865846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93043aef-4ea3-4e1b-a503-0489cebc8dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.866784 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93043aef-4ea3-4e1b-a503-0489cebc8dce admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.867063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93043aef-4ea3-4e1b-a503-0489cebc8dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.867348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93043aef-4ea3-4e1b-a503-0489cebc8dce admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.868170 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93043aef-4ea3-4e1b-a503-0489cebc8dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.870150 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93043aef-4ea3-4e1b-a503-0489cebc8dce admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:57.881525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 785/1554] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:05:57 2022] PUT /identity/v3/projects/09b63eccabbc498cb96dd4bbcef0969a/users/1438ef91441f4493844a915ce12fcfa4/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:05:57.883073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b532f48-b1cc-4b95-a4fb-3e3894ca29e9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.883418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b532f48-b1cc-4b95-a4fb-3e3894ca29e9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.883795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b532f48-b1cc-4b95-a4fb-3e3894ca29e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.906000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6b532f48-b1cc-4b95-a4fb-3e3894ca29e9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:57.910937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6b532f48-b1cc-4b95-a4fb-3e3894ca29e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:57.970958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 786/1555] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:57.972225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52561df-7c22-4578-aa11-9b1c643e0ab9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.972566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52561df-7c22-4578-aa11-9b1c643e0ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.972855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52561df-7c22-4578-aa11-9b1c643e0ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.975867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 770/1556] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:57.976659 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93043aef-4ea3-4e1b-a503-0489cebc8dce admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:57.982717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:05:57.986532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93043aef-4ea3-4e1b-a503-0489cebc8dce admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:57.991846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d7684-30ef-4e0c-9c73-310f1ed813d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:57.992493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7684-30ef-4e0c-9c73-310f1ed813d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:57.992781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7684-30ef-4e0c-9c73-310f1ed813d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:57.993089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7684-30ef-4e0c-9c73-310f1ed813d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:57.993816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d7684-30ef-4e0c-9c73-310f1ed813d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:57.994222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a52561df-7c22-4578-aa11-9b1c643e0ab9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:57.995399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a52561df-7c22-4578-aa11-9b1c643e0ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:57.995700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d7684-30ef-4e0c-9c73-310f1ed813d4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.003629 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 787/1557] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:58.006073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af1eae-ea04-4395-b28d-a7c14a6d83c2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.006361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af1eae-ea04-4395-b28d-a7c14a6d83c2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.006640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af1eae-ea04-4395-b28d-a7c14a6d83c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.022345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:05:58.027994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-05af1eae-ea04-4395-b28d-a7c14a6d83c2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:58.029165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-05af1eae-ea04-4395-b28d-a7c14a6d83c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:58.037095 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 788/1558] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:58.038441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25595a82-037a-4e95-9889-f9a431f5fda2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.038864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25595a82-037a-4e95-9889-f9a431f5fda2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.039140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25595a82-037a-4e95-9889-f9a431f5fda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.056411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-25595a82-037a-4e95-9889-f9a431f5fda2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:58.059934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-25595a82-037a-4e95-9889-f9a431f5fda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:58.065334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 789/1559] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:58.069402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9af59-5de6-4274-9746-278aa577b095 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.069402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9af59-5de6-4274-9746-278aa577b095 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.069402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9af59-5de6-4274-9746-278aa577b095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.085777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2bf9af59-5de6-4274-9746-278aa577b095 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:58.087206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2bf9af59-5de6-4274-9746-278aa577b095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:58.089179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 771/1560] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:05:57 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:58.089179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc7a7b-a988-4bc5-9452-4ab75e4b1f40 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.089736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc7a7b-a988-4bc5-9452-4ab75e4b1f40 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.089829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc7a7b-a988-4bc5-9452-4ab75e4b1f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.107570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fbfc7a7b-a988-4bc5-9452-4ab75e4b1f40 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:58.108770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fbfc7a7b-a988-4bc5-9452-4ab75e4b1f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:58.148577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 790/1561] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:58.149573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9af59-5de6-4274-9746-278aa577b095 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.158782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9af59-5de6-4274-9746-278aa577b095 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.163968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b38fb-2a6a-4fea-bbba-27075de69df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.164657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b38fb-2a6a-4fea-bbba-27075de69df6 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.164942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b38fb-2a6a-4fea-bbba-27075de69df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.165220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b38fb-2a6a-4fea-bbba-27075de69df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.165940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b38fb-2a6a-4fea-bbba-27075de69df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.167581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b38fb-2a6a-4fea-bbba-27075de69df6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.170484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 772/1562] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:58.171931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26915caa-69d3-4bd6-95ea-278116adf0f8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.172217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26915caa-69d3-4bd6-95ea-278116adf0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.172490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26915caa-69d3-4bd6-95ea-278116adf0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.182917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 791/1563] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:58.184420 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32b06b-c82a-4662-85c6-2729c43c7799 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.184709 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32b06b-c82a-4662-85c6-2729c43c7799 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.185040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32b06b-c82a-4662-85c6-2729c43c7799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.192580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-26915caa-69d3-4bd6-95ea-278116adf0f8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:58.193767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-26915caa-69d3-4bd6-95ea-278116adf0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:58.201531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 773/1564] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:58.203776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6e53-64ea-416d-8015-eb94cc419765 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.203908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5d32b06b-c82a-4662-85c6-2729c43c7799 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:58.204184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6e53-64ea-416d-8015-eb94cc419765 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.204536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6e53-64ea-416d-8015-eb94cc419765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.204944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5d32b06b-c82a-4662-85c6-2729c43c7799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:58.212878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 792/1565] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:58.213651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32b06b-c82a-4662-85c6-2729c43c7799 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.214902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5d32b06b-c82a-4662-85c6-2729c43c7799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:58.224273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4eeb6e53-64ea-416d-8015-eb94cc419765 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:58.225453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4eeb6e53-64ea-416d-8015-eb94cc419765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:58.233337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 774/1566] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:58.234120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb6e53-64ea-416d-8015-eb94cc419765 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.243755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb6e53-64ea-416d-8015-eb94cc419765 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.248277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac193eb0-daad-4774-bfaf-cd868bee141d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.248916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac193eb0-daad-4774-bfaf-cd868bee141d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.249199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac193eb0-daad-4774-bfaf-cd868bee141d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.249481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac193eb0-daad-4774-bfaf-cd868bee141d admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.250201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac193eb0-daad-4774-bfaf-cd868bee141d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.252102 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac193eb0-daad-4774-bfaf-cd868bee141d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.274839 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32b06b-c82a-4662-85c6-2729c43c7799 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.278329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe82fc-d6e4-4ecc-a781-ed8908b1169f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.278970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe82fc-d6e4-4ecc-a781-ed8908b1169f service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.279251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe82fc-d6e4-4ecc-a781-ed8908b1169f service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.279529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe82fc-d6e4-4ecc-a781-ed8908b1169f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.289038 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe82fc-d6e4-4ecc-a781-ed8908b1169f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.290497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe82fc-d6e4-4ecc-a781-ed8908b1169f service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.299757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 793/1567] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:05:58.301898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a233a77-8bfd-47e3-a03c-495c6df396a4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.302222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a233a77-8bfd-47e3-a03c-495c6df396a4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.302503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a233a77-8bfd-47e3-a03c-495c6df396a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.320143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3a233a77-8bfd-47e3-a03c-495c6df396a4 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:58.321334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3a233a77-8bfd-47e3-a03c-495c6df396a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:58.329133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 794/1568] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:58.330355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405b5be-6329-4665-9f4f-2f89a3fc7d69 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.330640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405b5be-6329-4665-9f4f-2f89a3fc7d69 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.330916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405b5be-6329-4665-9f4f-2f89a3fc7d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.348790 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b405b5be-6329-4665-9f4f-2f89a3fc7d69 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:58.349973 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b405b5be-6329-4665-9f4f-2f89a3fc7d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:58.358832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 795/1569] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:58.359830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405b5be-6329-4665-9f4f-2f89a3fc7d69 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.368250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 775/1570] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:58.369084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac193eb0-daad-4774-bfaf-cd868bee141d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.372532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405b5be-6329-4665-9f4f-2f89a3fc7d69 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.377719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf17589-35e1-46fb-af90-ead3b64a1b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.378405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf17589-35e1-46fb-af90-ead3b64a1b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.378516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac193eb0-daad-4774-bfaf-cd868bee141d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.378982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf17589-35e1-46fb-af90-ead3b64a1b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.379699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf17589-35e1-46fb-af90-ead3b64a1b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.380423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf17589-35e1-46fb-af90-ead3b64a1b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.382068 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf17589-35e1-46fb-af90-ead3b64a1b40 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.383324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fcfce-97a3-4288-a863-f48dcd55c632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.383980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fcfce-97a3-4288-a863-f48dcd55c632 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.384251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fcfce-97a3-4288-a863-f48dcd55c632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.384530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fcfce-97a3-4288-a863-f48dcd55c632 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.385258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fcfce-97a3-4288-a863-f48dcd55c632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:05:58.385551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fcfce-97a3-4288-a863-f48dcd55c632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.387145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fcfce-97a3-4288-a863-f48dcd55c632 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.394796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 796/1571] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:58.394796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf17589-35e1-46fb-af90-ead3b64a1b40 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.405739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf17589-35e1-46fb-af90-ead3b64a1b40 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.406976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 776/1572] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:05:58.411557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fcfce-97a3-4288-a863-f48dcd55c632 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.412263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac544a-45c6-4aba-b6c8-9242df877a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.412998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac544a-45c6-4aba-b6c8-9242df877a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.413375 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac544a-45c6-4aba-b6c8-9242df877a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.413721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac544a-45c6-4aba-b6c8-9242df877a9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.414598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac544a-45c6-4aba-b6c8-9242df877a9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:05:58.414886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac544a-45c6-4aba-b6c8-9242df877a9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.416507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac544a-45c6-4aba-b6c8-9242df877a9a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.422657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fcfce-97a3-4288-a863-f48dcd55c632 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.430011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ffe89-c9bd-44a0-9e0d-3067bdeafa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.430623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ffe89-c9bd-44a0-9e0d-3067bdeafa35 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.430874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ffe89-c9bd-44a0-9e0d-3067bdeafa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.431140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ffe89-c9bd-44a0-9e0d-3067bdeafa35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.438417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 797/1573] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:05:58.438570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ffe89-c9bd-44a0-9e0d-3067bdeafa35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:05:58.438570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ffe89-c9bd-44a0-9e0d-3067bdeafa35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.438759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac544a-45c6-4aba-b6c8-9242df877a9a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.438966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ffe89-c9bd-44a0-9e0d-3067bdeafa35 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.445620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac544a-45c6-4aba-b6c8-9242df877a9a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.446920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 777/1574] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:05:58.447740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ffe89-c9bd-44a0-9e0d-3067bdeafa35 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.450468 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce01e9-cabb-44b1-9e32-91e8b0be8e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.451123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce01e9-cabb-44b1-9e32-91e8b0be8e06 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.451566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce01e9-cabb-44b1-9e32-91e8b0be8e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.451848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce01e9-cabb-44b1-9e32-91e8b0be8e06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.452706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce01e9-cabb-44b1-9e32-91e8b0be8e06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:05:58.452994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce01e9-cabb-44b1-9e32-91e8b0be8e06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.454835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce01e9-cabb-44b1-9e32-91e8b0be8e06 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.458738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ffe89-c9bd-44a0-9e0d-3067bdeafa35 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.465033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a5a4c-b9db-474e-9067-4e5ff1a9dac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.466350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a5a4c-b9db-474e-9067-4e5ff1a9dac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.472685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a5a4c-b9db-474e-9067-4e5ff1a9dac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.472685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a5a4c-b9db-474e-9067-4e5ff1a9dac7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.472685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a5a4c-b9db-474e-9067-4e5ff1a9dac7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:05:58.472685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a5a4c-b9db-474e-9067-4e5ff1a9dac7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.473148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 798/1575] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:05:58.473148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce01e9-cabb-44b1-9e32-91e8b0be8e06 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.473320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a5a4c-b9db-474e-9067-4e5ff1a9dac7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.480123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce01e9-cabb-44b1-9e32-91e8b0be8e06 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.484680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 778/1576] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:05:58.485659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fd903-f0ad-4c63-a9e6-356635543351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.486376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fd903-f0ad-4c63-a9e6-356635543351 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.486657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fd903-f0ad-4c63-a9e6-356635543351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.486934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fd903-f0ad-4c63-a9e6-356635543351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.487635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a5a4c-b9db-474e-9067-4e5ff1a9dac7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.487751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fd903-f0ad-4c63-a9e6-356635543351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.491344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fd903-f0ad-4c63-a9e6-356635543351 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.500764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a5a4c-b9db-474e-9067-4e5ff1a9dac7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.503388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 799/1577] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:58.505750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fd903-f0ad-4c63-a9e6-356635543351 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.510271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90057bc7-c001-4395-8b65-cea88d5a2373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.511621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90057bc7-c001-4395-8b65-cea88d5a2373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.512216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90057bc7-c001-4395-8b65-cea88d5a2373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.512555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90057bc7-c001-4395-8b65-cea88d5a2373 admin admin] PATH_INFO: `/v3/projects/461a578ac4d44ffc9284d56a09d6c18e/users/9f6eb40fbef34931b668330b9f4db27d/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.518284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fd903-f0ad-4c63-a9e6-356635543351 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.523772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf0b03-6e19-4123-974b-e6495ba55313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.524429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0b03-6e19-4123-974b-e6495ba55313 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.528233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0b03-6e19-4123-974b-e6495ba55313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.528233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0b03-6e19-4123-974b-e6495ba55313 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.528233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf0b03-6e19-4123-974b-e6495ba55313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.528233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-abaf0b03-6e19-4123-974b-e6495ba55313 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:05:58.528233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 13 09:05:58.528233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: Jul 13 09:05:58.528233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf0b03-6e19-4123-974b-e6495ba55313 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.528837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90057bc7-c001-4395-8b65-cea88d5a2373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=461a578ac4d44ffc9284d56a09d6c18e, user_id=9f6eb40fbef34931b668330b9f4db27d, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.531118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90057bc7-c001-4395-8b65-cea88d5a2373 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.542231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:05:58.642876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 779/1578] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:05:58 2022] PUT /identity/v3/projects/461a578ac4d44ffc9284d56a09d6c18e/users/9f6eb40fbef34931b668330b9f4db27d/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:05:58.644222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90057bc7-c001-4395-8b65-cea88d5a2373 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.650905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 800/1579] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:58.650905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf0b03-6e19-4123-974b-e6495ba55313 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.710221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:05:58.726908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90057bc7-c001-4395-8b65-cea88d5a2373 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.731390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf0b03-6e19-4123-974b-e6495ba55313 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.731721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bf823-dd23-44f3-ad5a-7552a6cff657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.732380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bf823-dd23-44f3-ad5a-7552a6cff657 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.732661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bf823-dd23-44f3-ad5a-7552a6cff657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.732935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bf823-dd23-44f3-ad5a-7552a6cff657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.733401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828e9f0-4771-4d41-9d29-139e5c50d2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.733629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bf823-dd23-44f3-ad5a-7552a6cff657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.734212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828e9f0-4771-4d41-9d29-139e5c50d2df admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.734590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828e9f0-4771-4d41-9d29-139e5c50d2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.734932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828e9f0-4771-4d41-9d29-139e5c50d2df admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.735598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bf823-dd23-44f3-ad5a-7552a6cff657 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.735745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828e9f0-4771-4d41-9d29-139e5c50d2df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.737677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828e9f0-4771-4d41-9d29-139e5c50d2df admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.837290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 801/1580] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/projects => generated 393 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:58.840749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 780/1581] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/projects => generated 365 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:58.842431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bf823-dd23-44f3-ad5a-7552a6cff657 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.842572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828e9f0-4771-4d41-9d29-139e5c50d2df admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:58.923142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bf823-dd23-44f3-ad5a-7552a6cff657 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.923331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828e9f0-4771-4d41-9d29-139e5c50d2df admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:58.930696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a2b24-9f8c-4a66-a331-30d5496f6912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.931367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a2b24-9f8c-4a66-a331-30d5496f6912 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.931648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a2b24-9f8c-4a66-a331-30d5496f6912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.932353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e8d5e-e9c9-4e7d-908d-837e93eb087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:58.933078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e8d5e-e9c9-4e7d-908d-837e93eb087c admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:58.933440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e8d5e-e9c9-4e7d-908d-837e93eb087c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:58.933612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e8d5e-e9c9-4e7d-908d-837e93eb087c admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.934304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a2b24-9f8c-4a66-a331-30d5496f6912 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:58.934444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e8d5e-e9c9-4e7d-908d-837e93eb087c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.935089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a2b24-9f8c-4a66-a331-30d5496f6912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:58.936731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e8d5e-e9c9-4e7d-908d-837e93eb087c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:58.937407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a2b24-9f8c-4a66-a331-30d5496f6912 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.055061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 781/1582] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/projects => generated 381 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:59.058704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 802/1583] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:05:58 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:59.058704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d7cc2-2783-4a24-9028-ca799096c414 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.058704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d7cc2-2783-4a24-9028-ca799096c414 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.058704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d7cc2-2783-4a24-9028-ca799096c414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.060723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a2b24-9f8c-4a66-a331-30d5496f6912 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.077175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ce8d7cc2-2783-4a24-9028-ca799096c414 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:59.079145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ce8d7cc2-2783-4a24-9028-ca799096c414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:59.127613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a2b24-9f8c-4a66-a331-30d5496f6912 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.150363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 803/1584] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:05:59.152846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d7cc2-2783-4a24-9028-ca799096c414 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.166092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d7cc2-2783-4a24-9028-ca799096c414 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.171178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b6b4d-ff72-4134-884e-f6a5ec8899ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.172167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b6b4d-ff72-4134-884e-f6a5ec8899ef admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.172516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b6b4d-ff72-4134-884e-f6a5ec8899ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.172850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b6b4d-ff72-4134-884e-f6a5ec8899ef admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.173551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b6b4d-ff72-4134-884e-f6a5ec8899ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.176256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b6b4d-ff72-4134-884e-f6a5ec8899ef admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.183871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:05:59.198527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1d79-8eed-4ead-8825-5153af78332c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.199195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1d79-8eed-4ead-8825-5153af78332c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.199477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1d79-8eed-4ead-8825-5153af78332c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.199751 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1d79-8eed-4ead-8825-5153af78332c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.200496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd1d79-8eed-4ead-8825-5153af78332c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.202095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd1d79-8eed-4ead-8825-5153af78332c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.212029 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 782/1585] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.212940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1d79-8eed-4ead-8825-5153af78332c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.217771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:05:59.223249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1d79-8eed-4ead-8825-5153af78332c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.227827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4e02f-e183-4f57-8ac5-c4c8d884750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.228442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4e02f-e183-4f57-8ac5-c4c8d884750f admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.228720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4e02f-e183-4f57-8ac5-c4c8d884750f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.228994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4e02f-e183-4f57-8ac5-c4c8d884750f admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.229656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4e02f-e183-4f57-8ac5-c4c8d884750f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.231711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4e02f-e183-4f57-8ac5-c4c8d884750f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.299165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 804/1586] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:05:59 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:59.299165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b6b4d-ff72-4134-884e-f6a5ec8899ef admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.307054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b6b4d-ff72-4134-884e-f6a5ec8899ef admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.311606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa67e2-6b21-4052-bec3-97ad5cde644b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.312250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa67e2-6b21-4052-bec3-97ad5cde644b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.312535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa67e2-6b21-4052-bec3-97ad5cde644b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.312813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa67e2-6b21-4052-bec3-97ad5cde644b admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.313534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa67e2-6b21-4052-bec3-97ad5cde644b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.315519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa67e2-6b21-4052-bec3-97ad5cde644b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.349032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 783/1587] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:05:59 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:59.350543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4e02f-e183-4f57-8ac5-c4c8d884750f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.360395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4e02f-e183-4f57-8ac5-c4c8d884750f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.370605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9825ba-ba5d-4824-979f-a89e5309b96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.371346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9825ba-ba5d-4824-979f-a89e5309b96f admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.371629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9825ba-ba5d-4824-979f-a89e5309b96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.371902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9825ba-ba5d-4824-979f-a89e5309b96f admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.372626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9825ba-ba5d-4824-979f-a89e5309b96f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.374542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9825ba-ba5d-4824-979f-a89e5309b96f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.413613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 805/1588] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:05:59 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:59.414247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa67e2-6b21-4052-bec3-97ad5cde644b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.423938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa67e2-6b21-4052-bec3-97ad5cde644b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.430878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45d676-04b8-4631-97b5-fc0e0c26fb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.431535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45d676-04b8-4631-97b5-fc0e0c26fb0c admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.431820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45d676-04b8-4631-97b5-fc0e0c26fb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.432097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45d676-04b8-4631-97b5-fc0e0c26fb0c admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.432777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45d676-04b8-4631-97b5-fc0e0c26fb0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.434788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45d676-04b8-4631-97b5-fc0e0c26fb0c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.464765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 784/1589] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:05:59 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:59.465710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9825ba-ba5d-4824-979f-a89e5309b96f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.521924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9825ba-ba5d-4824-979f-a89e5309b96f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.523191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 806/1590] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:05:59 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:05:59.523982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45d676-04b8-4631-97b5-fc0e0c26fb0c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.528675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b567e0-a0d4-4fcf-b4b0-8d72cca18835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.529328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b567e0-a0d4-4fcf-b4b0-8d72cca18835 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.529620 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b567e0-a0d4-4fcf-b4b0-8d72cca18835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.529886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b567e0-a0d4-4fcf-b4b0-8d72cca18835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.533258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45d676-04b8-4631-97b5-fc0e0c26fb0c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.538592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bd4fa-4422-4fa4-8c26-da6d89364334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.539453 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bd4fa-4422-4fa4-8c26-da6d89364334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.539728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bd4fa-4422-4fa4-8c26-da6d89364334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.539880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b567e0-a0d4-4fcf-b4b0-8d72cca18835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.540030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bd4fa-4422-4fa4-8c26-da6d89364334 admin admin] PATH_INFO: `/v3/projects/537f3a6f839948638c78cd68ff9fcf24/users/3121e6920b424e01a44bf057d82ebcbc/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.541317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b567e0-a0d4-4fcf-b4b0-8d72cca18835 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.552771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 785/1591] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:05:59.553486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b567e0-a0d4-4fcf-b4b0-8d72cca18835 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.564573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b567e0-a0d4-4fcf-b4b0-8d72cca18835 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.570518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2a95e-1fdc-4dbc-9315-d868ef1c0188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.571311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2a95e-1fdc-4dbc-9315-d868ef1c0188 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.571536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2a95e-1fdc-4dbc-9315-d868ef1c0188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.571914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2a95e-1fdc-4dbc-9315-d868ef1c0188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.572796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2a95e-1fdc-4dbc-9315-d868ef1c0188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.575218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2a95e-1fdc-4dbc-9315-d868ef1c0188 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.584033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bd4fa-4422-4fa4-8c26-da6d89364334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=537f3a6f839948638c78cd68ff9fcf24, user_id=3121e6920b424e01a44bf057d82ebcbc, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.586829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bd4fa-4422-4fa4-8c26-da6d89364334 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.588163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 786/1592] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.589173 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2a95e-1fdc-4dbc-9315-d868ef1c0188 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.599969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2a95e-1fdc-4dbc-9315-d868ef1c0188 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.604743 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d2686-bce0-42e3-905d-2c7e959c4cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.612058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d2686-bce0-42e3-905d-2c7e959c4cad admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.612058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d2686-bce0-42e3-905d-2c7e959c4cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.612058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d2686-bce0-42e3-905d-2c7e959c4cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.612058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d2686-bce0-42e3-905d-2c7e959c4cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.612058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d2686-bce0-42e3-905d-2c7e959c4cad admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.619026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 787/1593] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.619656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d2686-bce0-42e3-905d-2c7e959c4cad admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.630062 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d2686-bce0-42e3-905d-2c7e959c4cad admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.635740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fe1e2-6f82-4482-902a-13ac67005f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.636518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fe1e2-6f82-4482-902a-13ac67005f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.636807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fe1e2-6f82-4482-902a-13ac67005f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.637082 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fe1e2-6f82-4482-902a-13ac67005f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.637858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fe1e2-6f82-4482-902a-13ac67005f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.639602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fe1e2-6f82-4482-902a-13ac67005f1b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.652174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 788/1594] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.652174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fe1e2-6f82-4482-902a-13ac67005f1b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.663986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fe1e2-6f82-4482-902a-13ac67005f1b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.669900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082463b2-f173-418a-8afe-b355a576731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.670718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082463b2-f173-418a-8afe-b355a576731b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.671191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082463b2-f173-418a-8afe-b355a576731b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.671480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082463b2-f173-418a-8afe-b355a576731b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.672372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082463b2-f173-418a-8afe-b355a576731b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.673939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082463b2-f173-418a-8afe-b355a576731b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.687449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 807/1595] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:05:59 2022] PUT /identity/v3/projects/537f3a6f839948638c78cd68ff9fcf24/users/3121e6920b424e01a44bf057d82ebcbc/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:05:59.687449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bd4fa-4422-4fa4-8c26-da6d89364334 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.687709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 789/1596] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.690977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082463b2-f173-418a-8afe-b355a576731b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.716777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082463b2-f173-418a-8afe-b355a576731b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.758811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bd4fa-4422-4fa4-8c26-da6d89364334 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.763818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00b082-5ae4-49c6-84a3-1fc000669c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.768834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00b082-5ae4-49c6-84a3-1fc000669c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.768834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00b082-5ae4-49c6-84a3-1fc000669c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.768834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00b082-5ae4-49c6-84a3-1fc000669c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.768834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00b082-5ae4-49c6-84a3-1fc000669c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.768834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00b082-5ae4-49c6-84a3-1fc000669c50 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.780852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 808/1597] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.784485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00b082-5ae4-49c6-84a3-1fc000669c50 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.786879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fe455-4d38-4da7-a7dd-fa01e5b12b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.788025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fe455-4d38-4da7-a7dd-fa01e5b12b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.788631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fe455-4d38-4da7-a7dd-fa01e5b12b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.789279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fe455-4d38-4da7-a7dd-fa01e5b12b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.790793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fe455-4d38-4da7-a7dd-fa01e5b12b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.793715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fe455-4d38-4da7-a7dd-fa01e5b12b26 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.796739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00b082-5ae4-49c6-84a3-1fc000669c50 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.802428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1821f-cdc1-4b83-b8cd-9500e9507ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.803293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1821f-cdc1-4b83-b8cd-9500e9507ead admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.803652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1821f-cdc1-4b83-b8cd-9500e9507ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.804054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1821f-cdc1-4b83-b8cd-9500e9507ead admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.804887 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1821f-cdc1-4b83-b8cd-9500e9507ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.806798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1821f-cdc1-4b83-b8cd-9500e9507ead admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.810593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 790/1598] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.811936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fe455-4d38-4da7-a7dd-fa01e5b12b26 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.822934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 809/1599] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.823867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1821f-cdc1-4b83-b8cd-9500e9507ead admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.830087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fe455-4d38-4da7-a7dd-fa01e5b12b26 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.835659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1821f-cdc1-4b83-b8cd-9500e9507ead admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.838676 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823ebb3-2413-401e-b471-df8c84525b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.839493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823ebb3-2413-401e-b471-df8c84525b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.839826 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823ebb3-2413-401e-b471-df8c84525b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.840208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823ebb3-2413-401e-b471-df8c84525b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.841012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823ebb3-2413-401e-b471-df8c84525b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.841631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c416a-f5ef-42ac-9cc7-58ca8e6e4d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.842483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c416a-f5ef-42ac-9cc7-58ca8e6e4d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.842868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c416a-f5ef-42ac-9cc7-58ca8e6e4d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.843266 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c416a-f5ef-42ac-9cc7-58ca8e6e4d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.843610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823ebb3-2413-401e-b471-df8c84525b92 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.844232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c416a-f5ef-42ac-9cc7-58ca8e6e4d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.846239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c416a-f5ef-42ac-9cc7-58ca8e6e4d42 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.859240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 810/1600] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.860963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034cdf7-28c9-4f8c-b450-2597121c74f3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.861359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034cdf7-28c9-4f8c-b450-2597121c74f3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.861749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034cdf7-28c9-4f8c-b450-2597121c74f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.862552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 791/1601] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.863486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823ebb3-2413-401e-b471-df8c84525b92 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.877261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823ebb3-2413-401e-b471-df8c84525b92 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.883413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfd60a-30a6-483f-a528-7c7a3e003849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.884325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfd60a-30a6-483f-a528-7c7a3e003849 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.885045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfd60a-30a6-483f-a528-7c7a3e003849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.885230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfd60a-30a6-483f-a528-7c7a3e003849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.885963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfd60a-30a6-483f-a528-7c7a3e003849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.888726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfd60a-30a6-483f-a528-7c7a3e003849 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.896765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:05:59.903760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a034cdf7-28c9-4f8c-b450-2597121c74f3 None None] MFA Rules not processed for user `3121e6920b424e01a44bf057d82ebcbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:05:59.904991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 792/1602] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:05:59.905570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a034cdf7-28c9-4f8c-b450-2597121c74f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:05:59.906300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfd60a-30a6-483f-a528-7c7a3e003849 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:05:59.920154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfd60a-30a6-483f-a528-7c7a3e003849 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:05:59.927507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5494bb0-6e1b-42c1-a97c-da8877b90117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:05:59.929074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5494bb0-6e1b-42c1-a97c-da8877b90117 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:05:59.929411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5494bb0-6e1b-42c1-a97c-da8877b90117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:05:59.930290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5494bb0-6e1b-42c1-a97c-da8877b90117 admin admin] PATH_INFO: `/v3/projects/eb9e230019f042dbbc680fc3b1340d41/users/35c7c83855084c188ab8a6dacb57a24b/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:05:59.948659 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5494bb0-6e1b-42c1-a97c-da8877b90117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9e230019f042dbbc680fc3b1340d41, user_id=35c7c83855084c188ab8a6dacb57a24b, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:05:59.952218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5494bb0-6e1b-42c1-a97c-da8877b90117 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:05:59.992906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=10,keystone:SELECT=90 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:00.008162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 811/1603] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:05:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:00.010300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034cdf7-28c9-4f8c-b450-2597121c74f3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:00.032604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034cdf7-28c9-4f8c-b450-2597121c74f3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:00.039190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a7454-2cfe-4419-b2d0-8e630507e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:00.040273 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a7454-2cfe-4419-b2d0-8e630507e95e admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.040556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a7454-2cfe-4419-b2d0-8e630507e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.040891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a7454-2cfe-4419-b2d0-8e630507e95e admin admin] PATH_INFO: `/v3/projects/7afbdee349034d2182240f1f7230236f/users/d9bffb24cf26486a9a8947882b52ec98/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.057346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 793/1604] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:05:59 2022] PUT /identity/v3/projects/eb9e230019f042dbbc680fc3b1340d41/users/35c7c83855084c188ab8a6dacb57a24b/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:00.058328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5494bb0-6e1b-42c1-a97c-da8877b90117 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:00.065306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a7454-2cfe-4419-b2d0-8e630507e95e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afbdee349034d2182240f1f7230236f, user_id=d9bffb24cf26486a9a8947882b52ec98, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:00.068933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a7454-2cfe-4419-b2d0-8e630507e95e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:00.125584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5494bb0-6e1b-42c1-a97c-da8877b90117 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:00.131304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667246db-7da2-4f06-98d9-f37c35a49f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:00.132207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667246db-7da2-4f06-98d9-f37c35a49f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.132487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667246db-7da2-4f06-98d9-f37c35a49f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.132765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667246db-7da2-4f06-98d9-f37c35a49f41 admin admin] PATH_INFO: `/v3/projects/ac97aef9caa54a8eb844b6c075c5ff48/users/5b4fb5ca6bd04466bc430c891cc40742/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.148546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667246db-7da2-4f06-98d9-f37c35a49f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac97aef9caa54a8eb844b6c075c5ff48, user_id=5b4fb5ca6bd04466bc430c891cc40742, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:00.153031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667246db-7da2-4f06-98d9-f37c35a49f41 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:00.179414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 812/1605] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:00 2022] PUT /identity/v3/projects/7afbdee349034d2182240f1f7230236f/users/d9bffb24cf26486a9a8947882b52ec98/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:00.187365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a7454-2cfe-4419-b2d0-8e630507e95e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:00.253622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a7454-2cfe-4419-b2d0-8e630507e95e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:00.264737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 794/1606] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:00 2022] PUT /identity/v3/projects/ac97aef9caa54a8eb844b6c075c5ff48/users/5b4fb5ca6bd04466bc430c891cc40742/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:00.277042 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedc132-cd4f-4dc6-a56c-7261e9429556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:00.277475 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667246db-7da2-4f06-98d9-f37c35a49f41 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:00.279466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedc132-cd4f-4dc6-a56c-7261e9429556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.280330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedc132-cd4f-4dc6-a56c-7261e9429556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.281032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedc132-cd4f-4dc6-a56c-7261e9429556 admin admin] PATH_INFO: `/v3/projects/c1bcbd0180e94d7eb8a4e1ed1eb48931/users/4853ac8be8164693ae16919a59a93ada/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.324772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedc132-cd4f-4dc6-a56c-7261e9429556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1bcbd0180e94d7eb8a4e1ed1eb48931, user_id=4853ac8be8164693ae16919a59a93ada, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:00.324772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedc132-cd4f-4dc6-a56c-7261e9429556 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:00.376566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667246db-7da2-4f06-98d9-f37c35a49f41 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:00.382023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e3dd8-a16e-48c3-a686-8c9e8474f37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:00.383050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e3dd8-a16e-48c3-a686-8c9e8474f37f admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.383481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e3dd8-a16e-48c3-a686-8c9e8474f37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.383824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e3dd8-a16e-48c3-a686-8c9e8474f37f admin admin] PATH_INFO: `/v3/projects/b9915e7547ac42f89570543def2962b2/users/3dcc00db9ed943beb658c7d09e66f9be/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.401126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e3dd8-a16e-48c3-a686-8c9e8474f37f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9915e7547ac42f89570543def2962b2, user_id=3dcc00db9ed943beb658c7d09e66f9be, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:00.403980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e3dd8-a16e-48c3-a686-8c9e8474f37f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:00.417957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 813/1607] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:00 2022] PUT /identity/v3/projects/c1bcbd0180e94d7eb8a4e1ed1eb48931/users/4853ac8be8164693ae16919a59a93ada/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:00.419474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64873eb-670b-4fe0-b0d8-aeeb1d11054d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.419763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64873eb-670b-4fe0-b0d8-aeeb1d11054d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.420035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64873eb-670b-4fe0-b0d8-aeeb1d11054d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.421572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 814/1608] 10.222.0.40 () {54 vars in 961 bytes} [Wed Jul 13 09:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:06:00.422642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddac40b-3d6d-4c20-92d8-6ed18b2cf3df None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.422994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddac40b-3d6d-4c20-92d8-6ed18b2cf3df None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.423251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddac40b-3d6d-4c20-92d8-6ed18b2cf3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.452538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2ddac40b-3d6d-4c20-92d8-6ed18b2cf3df None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:00.453738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2ddac40b-3d6d-4c20-92d8-6ed18b2cf3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:00.509271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 795/1609] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:00 2022] PUT /identity/v3/projects/b9915e7547ac42f89570543def2962b2/users/3dcc00db9ed943beb658c7d09e66f9be/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:00.510983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccb1fc-1216-4d24-b7f0-47084960d927 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.511333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccb1fc-1216-4d24-b7f0-47084960d927 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.511941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccb1fc-1216-4d24-b7f0-47084960d927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.532696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 815/1610] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:00.534378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785cd3f-932a-47b1-bc9f-d7041ed43a1d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.534964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785cd3f-932a-47b1-bc9f-d7041ed43a1d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.536006 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785cd3f-932a-47b1-bc9f-d7041ed43a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.539233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b9ccb1fc-1216-4d24-b7f0-47084960d927 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:00.541000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b9ccb1fc-1216-4d24-b7f0-47084960d927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:00.591735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0785cd3f-932a-47b1-bc9f-d7041ed43a1d None None] MFA Rules not processed for user `5b4fb5ca6bd04466bc430c891cc40742`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:00.593025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0785cd3f-932a-47b1-bc9f-d7041ed43a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:00.596509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:00.632068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 796/1611] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:00.636209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144ef0d-f163-4cf8-951d-d339634c2836 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.637627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144ef0d-f163-4cf8-951d-d339634c2836 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.637928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144ef0d-f163-4cf8-951d-d339634c2836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.695643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f144ef0d-f163-4cf8-951d-d339634c2836 None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:00.700688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f144ef0d-f163-4cf8-951d-d339634c2836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:00.758367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 816/1612] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:00.763409 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba975209-e0c4-43c6-903a-73db78c7ee7f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.763698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba975209-e0c4-43c6-903a-73db78c7ee7f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.764364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba975209-e0c4-43c6-903a-73db78c7ee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.800965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ba975209-e0c4-43c6-903a-73db78c7ee7f None None] MFA Rules not processed for user `4853ac8be8164693ae16919a59a93ada`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:00.802519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ba975209-e0c4-43c6-903a-73db78c7ee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:00.816407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 797/1613] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Jul 13 09:06:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:00.817896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614dde0-5fe0-4b4f-a873-7fcc841fe5e2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:00.818199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614dde0-5fe0-4b4f-a873-7fcc841fe5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:00.818480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614dde0-5fe0-4b4f-a873-7fcc841fe5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:00.828568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:00.858406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7614dde0-5fe0-4b4f-a873-7fcc841fe5e2 None None] MFA Rules not processed for user `3dcc00db9ed943beb658c7d09e66f9be`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:00.860715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7614dde0-5fe0-4b4f-a873-7fcc841fe5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:00.899787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 817/1614] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:00.899787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba975209-e0c4-43c6-903a-73db78c7ee7f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:01.008393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 798/1615] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:01.008393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614dde0-5fe0-4b4f-a873-7fcc841fe5e2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:01.032687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba975209-e0c4-43c6-903a-73db78c7ee7f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:01.039544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed64dd-70f3-405b-8fc1-e0f7ea07a444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:01.040750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed64dd-70f3-405b-8fc1-e0f7ea07a444 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.041600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed64dd-70f3-405b-8fc1-e0f7ea07a444 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.042235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed64dd-70f3-405b-8fc1-e0f7ea07a444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.074660 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed64dd-70f3-405b-8fc1-e0f7ea07a444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:01.084534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed64dd-70f3-405b-8fc1-e0f7ea07a444 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:01.097059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614dde0-5fe0-4b4f-a873-7fcc841fe5e2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:01.102357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 818/1616] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:01.103735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed64dd-70f3-405b-8fc1-e0f7ea07a444 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:01.106513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721671f-c0ea-4944-b4e2-34fda7b64ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:01.107450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721671f-c0ea-4944-b4e2-34fda7b64ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.107925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721671f-c0ea-4944-b4e2-34fda7b64ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.108366 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721671f-c0ea-4944-b4e2-34fda7b64ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.124119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721671f-c0ea-4944-b4e2-34fda7b64ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:01.125624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721671f-c0ea-4944-b4e2-34fda7b64ec0 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:01.130869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed64dd-70f3-405b-8fc1-e0f7ea07a444 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:01.139657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023f6e3-3e34-4d93-a9b9-b503b9ddb744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:01.140251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 799/1617] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:01.140739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023f6e3-3e34-4d93-a9b9-b503b9ddb744 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.141145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721671f-c0ea-4944-b4e2-34fda7b64ec0 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:01.141294 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023f6e3-3e34-4d93-a9b9-b503b9ddb744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.141996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023f6e3-3e34-4d93-a9b9-b503b9ddb744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.143037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023f6e3-3e34-4d93-a9b9-b503b9ddb744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:01.145480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023f6e3-3e34-4d93-a9b9-b503b9ddb744 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:01.157638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721671f-c0ea-4944-b4e2-34fda7b64ec0 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:01.165412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4db419-7f2e-4857-bf7d-020a4f633ed7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:01.166112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4db419-7f2e-4857-bf7d-020a4f633ed7 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.166452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4db419-7f2e-4857-bf7d-020a4f633ed7 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.166822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4db419-7f2e-4857-bf7d-020a4f633ed7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.223489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 819/1618] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:01 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:01.223826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023f6e3-3e34-4d93-a9b9-b503b9ddb744 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:01.231748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4db419-7f2e-4857-bf7d-020a4f633ed7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:01.232813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4db419-7f2e-4857-bf7d-020a4f633ed7 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:01.245207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 800/1619] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:01.247700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34486dac-8ee9-4fbd-97a9-0d6a373b882a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.248106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34486dac-8ee9-4fbd-97a9-0d6a373b882a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.248404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34486dac-8ee9-4fbd-97a9-0d6a373b882a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.250285 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 801/1620] 10.222.0.40 () {54 vars in 961 bytes} [Wed Jul 13 09:06:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:06:01.257542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34486dac-8ee9-4fbd-97a9-0d6a373b882a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:01.361420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34486dac-8ee9-4fbd-97a9-0d6a373b882a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:01.363892 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023f6e3-3e34-4d93-a9b9-b503b9ddb744 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:01.370786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fccb5-8fe9-43b9-a5a8-9b2232952c4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:01.372501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fccb5-8fe9-43b9-a5a8-9b2232952c4c service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.375615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fccb5-8fe9-43b9-a5a8-9b2232952c4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.375615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fccb5-8fe9-43b9-a5a8-9b2232952c4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.375950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fefeb8-07a5-4b4d-8de7-21ea25cc7220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:01.376651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fefeb8-07a5-4b4d-8de7-21ea25cc7220 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.376938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fefeb8-07a5-4b4d-8de7-21ea25cc7220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.377221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fefeb8-07a5-4b4d-8de7-21ea25cc7220 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.379153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fefeb8-07a5-4b4d-8de7-21ea25cc7220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:01.383796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fefeb8-07a5-4b4d-8de7-21ea25cc7220 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:01.489114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fccb5-8fe9-43b9-a5a8-9b2232952c4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:01.490812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fccb5-8fe9-43b9-a5a8-9b2232952c4c service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:01.514868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 802/1621] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:01 2022] POST /identity/v3/users => generated 296 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:01.516941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efeb18-8a82-4604-9289-86f2ac639651 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.517494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efeb18-8a82-4604-9289-86f2ac639651 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.518087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efeb18-8a82-4604-9289-86f2ac639651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.526311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 820/1622] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:01.527520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fccb5-8fe9-43b9-a5a8-9b2232952c4c service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:01.562022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fccb5-8fe9-43b9-a5a8-9b2232952c4c service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:01.563249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-84efeb18-8a82-4604-9289-86f2ac639651 None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:01.565874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-84efeb18-8a82-4604-9289-86f2ac639651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:01.572541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8809af0-1183-4fd8-a4f9-81e65c1dfca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:01.573391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8809af0-1183-4fd8-a4f9-81e65c1dfca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.574815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8809af0-1183-4fd8-a4f9-81e65c1dfca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.575152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8809af0-1183-4fd8-a4f9-81e65c1dfca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.577358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8809af0-1183-4fd8-a4f9-81e65c1dfca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:01.584538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:01.586259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8809af0-1183-4fd8-a4f9-81e65c1dfca2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:01.640342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 821/1623] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:01.642111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8809af0-1183-4fd8-a4f9-81e65c1dfca2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:01.676230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8809af0-1183-4fd8-a4f9-81e65c1dfca2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:01.693534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a20226-7449-4847-b617-ae48bd076137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:01.695119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a20226-7449-4847-b617-ae48bd076137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.695338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a20226-7449-4847-b617-ae48bd076137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.695618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a20226-7449-4847-b617-ae48bd076137 admin admin] PATH_INFO: `/v3/projects/68e7322d628e4ead894ec0c9fb33f55a/users/27f522688c2a4d558080ee7c5786ebfe/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.734308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a20226-7449-4847-b617-ae48bd076137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e7322d628e4ead894ec0c9fb33f55a, user_id=27f522688c2a4d558080ee7c5786ebfe, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:01.746051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a20226-7449-4847-b617-ae48bd076137 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:01.772752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 803/1624] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Jul 13 09:06:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:01.774759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ada397-aa69-4d85-a8c2-f672dc6933f1 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.775050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ada397-aa69-4d85-a8c2-f672dc6933f1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.775329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ada397-aa69-4d85-a8c2-f672dc6933f1 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.776972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 804/1625] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 13 09:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:06:01.784194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4a24e-bc8a-43a9-970b-8d383fbb61bf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:01.785698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4a24e-bc8a-43a9-970b-8d383fbb61bf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:01.785698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4a24e-bc8a-43a9-970b-8d383fbb61bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:01.817293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e8e4a24e-bc8a-43a9-970b-8d383fbb61bf None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:01.818614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e8e4a24e-bc8a-43a9-970b-8d383fbb61bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:01.852807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 822/1626] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:01 2022] PUT /identity/v3/projects/68e7322d628e4ead894ec0c9fb33f55a/users/27f522688c2a4d558080ee7c5786ebfe/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:01.853026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 805/1627] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:06:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:01.853155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a20226-7449-4847-b617-ae48bd076137 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:01.855433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4a24e-bc8a-43a9-970b-8d383fbb61bf None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:01.883494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4a24e-bc8a-43a9-970b-8d383fbb61bf None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:02.029781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a20226-7449-4847-b617-ae48bd076137 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:02.036854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e13ee-eaa0-4252-a64d-0320fcae27f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:02.037897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e13ee-eaa0-4252-a64d-0320fcae27f8 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:02.038298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e13ee-eaa0-4252-a64d-0320fcae27f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:02.040724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d457e-0f37-48bd-aacb-d40273cb8a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:02.041610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d457e-0f37-48bd-aacb-d40273cb8a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:02.042084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d457e-0f37-48bd-aacb-d40273cb8a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:02.042529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d457e-0f37-48bd-aacb-d40273cb8a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:02.043260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d457e-0f37-48bd-aacb-d40273cb8a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:02.044877 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d457e-0f37-48bd-aacb-d40273cb8a4d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:02.045743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e13ee-eaa0-4252-a64d-0320fcae27f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:02.069628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 806/1628] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:02.070716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d457e-0f37-48bd-aacb-d40273cb8a4d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:02.115920 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:02.170531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e13ee-eaa0-4252-a64d-0320fcae27f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:02.173187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e13ee-eaa0-4252-a64d-0320fcae27f8 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:02.184513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 823/1629] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:02.185317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e13ee-eaa0-4252-a64d-0320fcae27f8 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:02.196883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e13ee-eaa0-4252-a64d-0320fcae27f8 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:02.202693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162159dc-0304-4eb9-bd0b-f4a6f8a2c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:02.203868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162159dc-0304-4eb9-bd0b-f4a6f8a2c00f admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:02.204236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162159dc-0304-4eb9-bd0b-f4a6f8a2c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:02.204570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162159dc-0304-4eb9-bd0b-f4a6f8a2c00f admin admin] PATH_INFO: `/v3/projects/68e7322d628e4ead894ec0c9fb33f55a/users/27f522688c2a4d558080ee7c5786ebfe/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:02.207332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162159dc-0304-4eb9-bd0b-f4a6f8a2c00f admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e7322d628e4ead894ec0c9fb33f55a, user_id=27f522688c2a4d558080ee7c5786ebfe, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:02.210620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162159dc-0304-4eb9-bd0b-f4a6f8a2c00f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:02.226764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d457e-0f37-48bd-aacb-d40273cb8a4d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:02.234500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 824/1630] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:02 2022] PUT /identity/v3/projects/68e7322d628e4ead894ec0c9fb33f55a/users/27f522688c2a4d558080ee7c5786ebfe/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:02.240325 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed21f7-27c8-4998-a042-f2eb9077e8d1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:02.241995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed21f7-27c8-4998-a042-f2eb9077e8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:02.243520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed21f7-27c8-4998-a042-f2eb9077e8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:02.312051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6fed21f7-27c8-4998-a042-f2eb9077e8d1 None None] MFA Rules not processed for user `27f522688c2a4d558080ee7c5786ebfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:02.313614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6fed21f7-27c8-4998-a042-f2eb9077e8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:02.327521 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:02.348289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa53798-2bb3-4ede-8cbb-bbc859251794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:02.349405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa53798-2bb3-4ede-8cbb-bbc859251794 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:02.349673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa53798-2bb3-4ede-8cbb-bbc859251794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:02.349951 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa53798-2bb3-4ede-8cbb-bbc859251794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:02.437185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 825/1631] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:02.444430 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa53798-2bb3-4ede-8cbb-bbc859251794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:02.446084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa53798-2bb3-4ede-8cbb-bbc859251794 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:02.462841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 807/1632] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:02.873013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5653c1b-6132-47b2-934a-f8038d5b1310 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:02.873331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5653c1b-6132-47b2-934a-f8038d5b1310 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:02.873578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5653c1b-6132-47b2-934a-f8038d5b1310 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:02.875234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 826/1633] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 13 09:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:06:02.883275 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbda86-c0d4-4624-8a0a-6326efbcac65 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:02.883574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbda86-c0d4-4624-8a0a-6326efbcac65 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:02.883852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbda86-c0d4-4624-8a0a-6326efbcac65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:02.915293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6abbda86-c0d4-4624-8a0a-6326efbcac65 None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:02.916510 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6abbda86-c0d4-4624-8a0a-6326efbcac65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:02.945923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 808/1634] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:06:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:02.994018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5653c1b-6132-47b2-934a-f8038d5b1310 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:03.012885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5653c1b-6132-47b2-934a-f8038d5b1310 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:03.021862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784d988-6aaf-4663-a268-05f6aef89fe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:03.022866 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784d988-6aaf-4663-a268-05f6aef89fe4 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:03.023355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784d988-6aaf-4663-a268-05f6aef89fe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:03.024300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784d988-6aaf-4663-a268-05f6aef89fe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:03.040999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784d988-6aaf-4663-a268-05f6aef89fe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:03.043795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784d988-6aaf-4663-a268-05f6aef89fe4 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:03.060961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 827/1635] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:03.201636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbda86-c0d4-4624-8a0a-6326efbcac65 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:03.213887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbda86-c0d4-4624-8a0a-6326efbcac65 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:03.232105 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70571809-4540-4ec2-8c49-d385a97099df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:03.233728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70571809-4540-4ec2-8c49-d385a97099df service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:03.234286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70571809-4540-4ec2-8c49-d385a97099df service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:03.234741 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70571809-4540-4ec2-8c49-d385a97099df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:03.505194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70571809-4540-4ec2-8c49-d385a97099df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:03.505194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70571809-4540-4ec2-8c49-d385a97099df service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:03.505194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 809/1636] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:03.509715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:03.716876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:03.724851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784d988-6aaf-4663-a268-05f6aef89fe4 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:03.743253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784d988-6aaf-4663-a268-05f6aef89fe4 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:03.770667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a31ae-2626-4208-b748-d5f92a60a154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:03.771456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a31ae-2626-4208-b748-d5f92a60a154 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:03.771818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a31ae-2626-4208-b748-d5f92a60a154 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:03.773195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a31ae-2626-4208-b748-d5f92a60a154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:03.881301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70571809-4540-4ec2-8c49-d385a97099df service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:04.027161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a31ae-2626-4208-b748-d5f92a60a154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:04.029864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a31ae-2626-4208-b748-d5f92a60a154 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:04.108868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 828/1637] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:04.137572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70571809-4540-4ec2-8c49-d385a97099df service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:04.155723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:04.156635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:04.156900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:04.157176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] PATH_INFO: `/v3/users/3dcc00db9ed943beb658c7d09e66f9be` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:04.165876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dcc00db9ed943beb658c7d09e66f9be)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:04.167935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:04.524579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dcc00db9ed943beb658c7d09e66f9be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:04.546120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dcc00db9ed943beb658c7d09e66f9be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:04.570840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dcc00db9ed943beb658c7d09e66f9be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:04.703215 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dcc00db9ed943beb658c7d09e66f9be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:04.704365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dcc00db9ed943beb658c7d09e66f9be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:04.720159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 810/1638] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:03 2022] DELETE /identity/v3/users/3dcc00db9ed943beb658c7d09e66f9be => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:04.726240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e234908-c26c-4f25-9e6e-05fa009a1a43 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:04.727202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e234908-c26c-4f25-9e6e-05fa009a1a43 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:04.727677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e234908-c26c-4f25-9e6e-05fa009a1a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:04.773802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8e234908-c26c-4f25-9e6e-05fa009a1a43 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:04.783505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8e234908-c26c-4f25-9e6e-05fa009a1a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:05.016797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 829/1639] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:05.048708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:05.051672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:05.172264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08c3cd-6330-441a-8a47-d025f70aa7dd admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:05.199976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cac9f4-5e22-4e22-bb1b-51ec3ff80108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:05.201686 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cac9f4-5e22-4e22-bb1b-51ec3ff80108 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:05.201892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cac9f4-5e22-4e22-bb1b-51ec3ff80108 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:05.202192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cac9f4-5e22-4e22-bb1b-51ec3ff80108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:05.228103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cac9f4-5e22-4e22-bb1b-51ec3ff80108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:05.232359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cac9f4-5e22-4e22-bb1b-51ec3ff80108 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:05.260025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 811/1640] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:06.780537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e234908-c26c-4f25-9e6e-05fa009a1a43 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:06.782254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8e234908-c26c-4f25-9e6e-05fa009a1a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:06.787921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ab293-5cc5-4236-95f9-5baa5f6e0fa3 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:06.788359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ab293-5cc5-4236-95f9-5baa5f6e0fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:06.788622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ab293-5cc5-4236-95f9-5baa5f6e0fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:06.795125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:06.816699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e234908-c26c-4f25-9e6e-05fa009a1a43 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:06.840325 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:06.842430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:06.843316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:06.844632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] PATH_INFO: `/v3/projects/b9915e7547ac42f89570543def2962b2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:06.849027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9915e7547ac42f89570543def2962b2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:06.860676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:06.865547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:06.873334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d59ab293-5cc5-4236-95f9-5baa5f6e0fa3 None None] MFA Rules not processed for user `1438ef91441f4493844a915ce12fcfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:06.876279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d59ab293-5cc5-4236-95f9-5baa5f6e0fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:07.042436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 812/1641] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:07.068759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ab293-5cc5-4236-95f9-5baa5f6e0fa3 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:07.070514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d59ab293-5cc5-4236-95f9-5baa5f6e0fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:07.076624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9915e7547ac42f89570543def2962b2', 'request_id': 'req-5bb854c4-70df-4699-832a-864780e2d3b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:07.091504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9915e7547ac42f89570543def2962b2', 'request_id': 'req-5bb854c4-70df-4699-832a-864780e2d3b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:07.109536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9915e7547ac42f89570543def2962b2', 'request_id': 'req-5bb854c4-70df-4699-832a-864780e2d3b3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:07.202153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ab293-5cc5-4236-95f9-5baa5f6e0fa3 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:07.212777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] The token cache is being invalidate because project b9915e7547ac42f89570543def2962b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:07.213073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:07.213721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5bb854c4-70df-4699-832a-864780e2d3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:07.215592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 830/1642] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:06 2022] DELETE /identity/v3/projects/b9915e7547ac42f89570543def2962b2 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:07.220320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6548c-764e-4a98-aa27-0a445ce88a1c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:07.220599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6548c-764e-4a98-aa27-0a445ce88a1c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:07.221050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6548c-764e-4a98-aa27-0a445ce88a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:07.259532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2ad6548c-764e-4a98-aa27-0a445ce88a1c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:07.263312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2ad6548c-764e-4a98-aa27-0a445ce88a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:07.411965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 831/1643] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:07.419807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659c34-0950-4b2a-8dd3-34c5499fb21b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:07.420324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659c34-0950-4b2a-8dd3-34c5499fb21b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:07.420770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659c34-0950-4b2a-8dd3-34c5499fb21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:07.456155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1a080-4bbf-4246-adf5-e87098bb7aab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:07.459565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-33659c34-0950-4b2a-8dd3-34c5499fb21b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:07.460903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-33659c34-0950-4b2a-8dd3-34c5499fb21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:07.461833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1a080-4bbf-4246-adf5-e87098bb7aab service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:07.462368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1a080-4bbf-4246-adf5-e87098bb7aab service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:07.462665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1a080-4bbf-4246-adf5-e87098bb7aab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:07.465898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7ac1a080-4bbf-4246-adf5-e87098bb7aab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:07.483304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 832/1644] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:07.493730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33659c34-0950-4b2a-8dd3-34c5499fb21b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:07.556261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33659c34-0950-4b2a-8dd3-34c5499fb21b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:07.574598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce463650-accc-434a-9152-be014efd2447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:07.575720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce463650-accc-434a-9152-be014efd2447 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:07.579125 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce463650-accc-434a-9152-be014efd2447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:07.580033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce463650-accc-434a-9152-be014efd2447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:07.582701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce463650-accc-434a-9152-be014efd2447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:06:07.583255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce463650-accc-434a-9152-be014efd2447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:07.587551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce463650-accc-434a-9152-be014efd2447 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:07.630578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 833/1645] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:06:07.633173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce463650-accc-434a-9152-be014efd2447 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:07.661131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce463650-accc-434a-9152-be014efd2447 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:07.675555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646adcd4-6361-4e0c-8e97-5a64f4f8aa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:07.677045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646adcd4-6361-4e0c-8e97-5a64f4f8aa88 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:07.678815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646adcd4-6361-4e0c-8e97-5a64f4f8aa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:07.679212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646adcd4-6361-4e0c-8e97-5a64f4f8aa88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:07.680740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646adcd4-6361-4e0c-8e97-5a64f4f8aa88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:07.688161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646adcd4-6361-4e0c-8e97-5a64f4f8aa88 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:07.689993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1a080-4bbf-4246-adf5-e87098bb7aab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:07.692705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1a080-4bbf-4246-adf5-e87098bb7aab service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:07.738709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 813/1646] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:07.864549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1a080-4bbf-4246-adf5-e87098bb7aab service nova] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:07.870483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 834/1647] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:07 2022] POST /identity/v3/projects => generated 403 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:07.883986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1a080-4bbf-4246-adf5-e87098bb7aab service nova] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:08.046662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527d719-2a54-4bcf-ae8c-c20f4e7628c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:08.047350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527d719-2a54-4bcf-ae8c-c20f4e7628c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:08.047879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527d719-2a54-4bcf-ae8c-c20f4e7628c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:08.048237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527d719-2a54-4bcf-ae8c-c20f4e7628c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:08.048935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527d719-2a54-4bcf-ae8c-c20f4e7628c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:08.051735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527d719-2a54-4bcf-ae8c-c20f4e7628c9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:08.156807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 814/1648] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:07 2022] POST /identity/v3/users => generated 302 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:06:08.161529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646adcd4-6361-4e0c-8e97-5a64f4f8aa88 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:08.185056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646adcd4-6361-4e0c-8e97-5a64f4f8aa88 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:08.217255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb507ded-87f9-4e18-8f82-8400298eaaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:08.218173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb507ded-87f9-4e18-8f82-8400298eaaee admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:08.218459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb507ded-87f9-4e18-8f82-8400298eaaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:08.218855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb507ded-87f9-4e18-8f82-8400298eaaee admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:08.219479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb507ded-87f9-4e18-8f82-8400298eaaee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:08.221292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb507ded-87f9-4e18-8f82-8400298eaaee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:08.248284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 835/1649] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:06:08.249632 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527d719-2a54-4bcf-ae8c-c20f4e7628c9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:08.288713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527d719-2a54-4bcf-ae8c-c20f4e7628c9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:08.300557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f794fe9-a82d-4d54-b710-85dbb61592a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:08.304078 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f794fe9-a82d-4d54-b710-85dbb61592a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:08.304535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f794fe9-a82d-4d54-b710-85dbb61592a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:08.305578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f794fe9-a82d-4d54-b710-85dbb61592a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:08.306750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f794fe9-a82d-4d54-b710-85dbb61592a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:08.310058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f794fe9-a82d-4d54-b710-85dbb61592a3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:08.334780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 815/1650] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:06:08.339063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb507ded-87f9-4e18-8f82-8400298eaaee admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:08.354666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb507ded-87f9-4e18-8f82-8400298eaaee admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:08.385693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d47b6f-2a0f-4800-8c9a-72d168112073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:08.386842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d47b6f-2a0f-4800-8c9a-72d168112073 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:08.387275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d47b6f-2a0f-4800-8c9a-72d168112073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:08.387760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d47b6f-2a0f-4800-8c9a-72d168112073 admin admin] PATH_INFO: `/v3/projects/c8907e8fe5704903835639a7a20b008d/users/cef9f25657744847aac4ef86f34b12ed/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:08.421319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d47b6f-2a0f-4800-8c9a-72d168112073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8907e8fe5704903835639a7a20b008d, user_id=cef9f25657744847aac4ef86f34b12ed, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:08.424845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d47b6f-2a0f-4800-8c9a-72d168112073 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:08.557432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 836/1651] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:08 2022] PUT /identity/v3/projects/c8907e8fe5704903835639a7a20b008d/users/cef9f25657744847aac4ef86f34b12ed/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:08.558239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb75a3-f3a9-4dad-9688-254a26137ac8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:08.559381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb75a3-f3a9-4dad-9688-254a26137ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:08.559666 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb75a3-f3a9-4dad-9688-254a26137ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:08.605011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bfeb75a3-f3a9-4dad-9688-254a26137ac8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:08.608443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bfeb75a3-f3a9-4dad-9688-254a26137ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:08.736094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 816/1652] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:08.757038 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d47b6f-2a0f-4800-8c9a-72d168112073 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:08.759072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-51d47b6f-2a0f-4800-8c9a-72d168112073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:08.795933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:08.893878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d47b6f-2a0f-4800-8c9a-72d168112073 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:08.911612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a30ed7-062b-43aa-86d4-b10e40a2b70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:08.913346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a30ed7-062b-43aa-86d4-b10e40a2b70d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:08.913839 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a30ed7-062b-43aa-86d4-b10e40a2b70d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:08.914378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a30ed7-062b-43aa-86d4-b10e40a2b70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:08.965886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a30ed7-062b-43aa-86d4-b10e40a2b70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:08.969082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a30ed7-062b-43aa-86d4-b10e40a2b70d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:09.030756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 837/1653] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:09.219102 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:13.663439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933111f-c2c7-4c93-a67e-ca93e4ad052c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:13.664381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933111f-c2c7-4c93-a67e-ca93e4ad052c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:13.664802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933111f-c2c7-4c93-a67e-ca93e4ad052c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:13.740977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a933111f-c2c7-4c93-a67e-ca93e4ad052c None None] MFA Rules not processed for user `cef9f25657744847aac4ef86f34b12ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:13.742264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a933111f-c2c7-4c93-a67e-ca93e4ad052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:13.841165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:13.912480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 817/1654] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:13.915555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a30ed7-062b-43aa-86d4-b10e40a2b70d service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:13.934892 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a30ed7-062b-43aa-86d4-b10e40a2b70d service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:13.951752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de69d45-40b1-4d64-ae10-1c1befe511cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:13.952436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de69d45-40b1-4d64-ae10-1c1befe511cb admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:13.952723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de69d45-40b1-4d64-ae10-1c1befe511cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:13.953080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de69d45-40b1-4d64-ae10-1c1befe511cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:13.953799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de69d45-40b1-4d64-ae10-1c1befe511cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:13.955808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de69d45-40b1-4d64-ae10-1c1befe511cb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:14.122562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 838/1655] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:13 2022] POST /identity/v3/projects => generated 405 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:14.123693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933111f-c2c7-4c93-a67e-ca93e4ad052c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:14.146648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933111f-c2c7-4c93-a67e-ca93e4ad052c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:14.280071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13c7a7-1a17-4700-bc3f-85d319c18a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:14.280774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13c7a7-1a17-4700-bc3f-85d319c18a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:14.281056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13c7a7-1a17-4700-bc3f-85d319c18a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:14.281329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13c7a7-1a17-4700-bc3f-85d319c18a94 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:14.282397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13c7a7-1a17-4700-bc3f-85d319c18a94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:14.284498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13c7a7-1a17-4700-bc3f-85d319c18a94 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:14.414998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 818/1656] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:14 2022] POST /identity/v3/users => generated 309 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:14.416894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de69d45-40b1-4d64-ae10-1c1befe511cb admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:14.452821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de69d45-40b1-4d64-ae10-1c1befe511cb admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:14.462973 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c6c96-1306-4a5c-9231-1e72d08147f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:14.463664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c6c96-1306-4a5c-9231-1e72d08147f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:14.463957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c6c96-1306-4a5c-9231-1e72d08147f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:14.464221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c6c96-1306-4a5c-9231-1e72d08147f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:14.464953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c6c96-1306-4a5c-9231-1e72d08147f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:14.466617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c6c96-1306-4a5c-9231-1e72d08147f3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:14.489280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 839/1657] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:14.494403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13c7a7-1a17-4700-bc3f-85d319c18a94 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:14.515973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13c7a7-1a17-4700-bc3f-85d319c18a94 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:14.548178 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526ce7b-5c03-4489-95c6-b68d4abdd151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:14.550765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526ce7b-5c03-4489-95c6-b68d4abdd151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:14.561953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526ce7b-5c03-4489-95c6-b68d4abdd151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:14.561953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526ce7b-5c03-4489-95c6-b68d4abdd151 admin admin] PATH_INFO: `/v3/projects/03dfa33ae39d4d5e8c2d7d2db7735070/users/fda6491fd5a144f687f86d9e1f7a7eed/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:14.563760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61aabb7-dec9-4955-968c-aa3053a71c8e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:14.564052 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61aabb7-dec9-4955-968c-aa3053a71c8e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:14.564472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61aabb7-dec9-4955-968c-aa3053a71c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:14.601138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e526ce7b-5c03-4489-95c6-b68d4abdd151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03dfa33ae39d4d5e8c2d7d2db7735070, user_id=fda6491fd5a144f687f86d9e1f7a7eed, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:14.608886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e526ce7b-5c03-4489-95c6-b68d4abdd151 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:14.632096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c61aabb7-dec9-4955-968c-aa3053a71c8e None None] MFA Rules not processed for user `9f6eb40fbef34931b668330b9f4db27d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:14.633354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c61aabb7-dec9-4955-968c-aa3053a71c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:14.711693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 819/1658] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:14 2022] PUT /identity/v3/projects/03dfa33ae39d4d5e8c2d7d2db7735070/users/fda6491fd5a144f687f86d9e1f7a7eed/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:14.716381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526ce7b-5c03-4489-95c6-b68d4abdd151 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:14.803082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 840/1659] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:14.817056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d6d3-e6f0-4246-b147-d16aeaafa757 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:14.817350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d6d3-e6f0-4246-b147-d16aeaafa757 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:14.817656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d6d3-e6f0-4246-b147-d16aeaafa757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:14.841471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526ce7b-5c03-4489-95c6-b68d4abdd151 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:14.847337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fce002-ed0a-4221-a6a6-e695808a3500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:14.848470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fce002-ed0a-4221-a6a6-e695808a3500 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:14.848930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fce002-ed0a-4221-a6a6-e695808a3500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:14.852556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fce002-ed0a-4221-a6a6-e695808a3500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:14.852556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fce002-ed0a-4221-a6a6-e695808a3500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:14.852556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fce002-ed0a-4221-a6a6-e695808a3500 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:14.861290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-aa42d6d3-e6f0-4246-b147-d16aeaafa757 None None] MFA Rules not processed for user `9f6eb40fbef34931b668330b9f4db27d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:14.865658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-aa42d6d3-e6f0-4246-b147-d16aeaafa757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:14.869728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 820/1660] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:14.871065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fce002-ed0a-4221-a6a6-e695808a3500 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:14.899593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fce002-ed0a-4221-a6a6-e695808a3500 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:14.921189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6266e-e39a-4412-8565-23154bb2e1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:14.922139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6266e-e39a-4412-8565-23154bb2e1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:14.922399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6266e-e39a-4412-8565-23154bb2e1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:14.922673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6266e-e39a-4412-8565-23154bb2e1e9 admin admin] PATH_INFO: `/v3/projects/03dfa33ae39d4d5e8c2d7d2db7735070/users/fda6491fd5a144f687f86d9e1f7a7eed/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:14.927770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6266e-e39a-4412-8565-23154bb2e1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03dfa33ae39d4d5e8c2d7d2db7735070, user_id=fda6491fd5a144f687f86d9e1f7a7eed, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:14.935305 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6266e-e39a-4412-8565-23154bb2e1e9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:15.008349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 821/1661] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:14 2022] PUT /identity/v3/projects/03dfa33ae39d4d5e8c2d7d2db7735070/users/fda6491fd5a144f687f86d9e1f7a7eed/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:15.086879 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 841/1662] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:15.099200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6266e-e39a-4412-8565-23154bb2e1e9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:15.103254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0da6266e-e39a-4412-8565-23154bb2e1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:15.197160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:15.330815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6266e-e39a-4412-8565-23154bb2e1e9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:15.369974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac961c0-6c5b-4095-a546-ffc87c6aa4a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:15.370675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac961c0-6c5b-4095-a546-ffc87c6aa4a3 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:15.370961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac961c0-6c5b-4095-a546-ffc87c6aa4a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:15.371225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac961c0-6c5b-4095-a546-ffc87c6aa4a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:15.538710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac961c0-6c5b-4095-a546-ffc87c6aa4a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:15.540263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac961c0-6c5b-4095-a546-ffc87c6aa4a3 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:15.587449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 822/1663] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:15.856846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42d6d3-e6f0-4246-b147-d16aeaafa757 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:15.859074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-aa42d6d3-e6f0-4246-b147-d16aeaafa757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:15.881804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42d6d3-e6f0-4246-b147-d16aeaafa757 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:15.917101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:16.033135 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:16.035209 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:16.035994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:16.036361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] PATH_INFO: `/v3/users/3121e6920b424e01a44bf057d82ebcbc` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:16.037826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3121e6920b424e01a44bf057d82ebcbc)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:16.039213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:06:16.039213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 13 09:06:16.039213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: Jul 13 09:06:16.049016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:16.405156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3121e6920b424e01a44bf057d82ebcbc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:16.419505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3121e6920b424e01a44bf057d82ebcbc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:16.493589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3121e6920b424e01a44bf057d82ebcbc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:16.494552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3121e6920b424e01a44bf057d82ebcbc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:16.495193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6eae8b3d-5bf9-46c4-87f9-23c69a403836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3121e6920b424e01a44bf057d82ebcbc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:16.531178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 842/1664] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:15 2022] DELETE /identity/v3/users/3121e6920b424e01a44bf057d82ebcbc => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:16.535960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ffce6-6d58-4433-895d-d7b79602f5a0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:16.536475 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ffce6-6d58-4433-895d-d7b79602f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:16.536679 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ffce6-6d58-4433-895d-d7b79602f5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:16.568493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bc7ffce6-6d58-4433-895d-d7b79602f5a0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:16.570465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bc7ffce6-6d58-4433-895d-d7b79602f5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:16.685808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 823/1665] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:16.707871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6d941-853a-4e99-81df-067481acdc40 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:16.708099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6d941-853a-4e99-81df-067481acdc40 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:16.711353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6d941-853a-4e99-81df-067481acdc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:16.714579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ffce6-6d58-4433-895d-d7b79602f5a0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:16.716174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bc7ffce6-6d58-4433-895d-d7b79602f5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:16.742235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1de6d941-853a-4e99-81df-067481acdc40 None None] MFA Rules not processed for user `1438ef91441f4493844a915ce12fcfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:16.745735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1de6d941-853a-4e99-81df-067481acdc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:16.841277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ffce6-6d58-4433-895d-d7b79602f5a0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:16.859940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cbebe-91d7-4b4f-8b86-40fb0f361ddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:16.863568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cbebe-91d7-4b4f-8b86-40fb0f361ddb service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:16.863944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cbebe-91d7-4b4f-8b86-40fb0f361ddb service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:16.864256 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cbebe-91d7-4b4f-8b86-40fb0f361ddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:16.887149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 843/1666] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:16.899230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cbebe-91d7-4b4f-8b86-40fb0f361ddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:16.900864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cbebe-91d7-4b4f-8b86-40fb0f361ddb service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:16.917924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 824/1667] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:17.033489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a24ef1-9ebb-470d-8a15-60fb2a1dc931 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:17.033782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a24ef1-9ebb-470d-8a15-60fb2a1dc931 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:17.034214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a24ef1-9ebb-470d-8a15-60fb2a1dc931 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:17.035863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 844/1668] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 13 09:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:06:17.042120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba7d38-d758-4998-83ef-8c206e58ebec None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:17.042507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba7d38-d758-4998-83ef-8c206e58ebec None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:17.042860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba7d38-d758-4998-83ef-8c206e58ebec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:17.061912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:17.138798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-18ba7d38-d758-4998-83ef-8c206e58ebec None None] MFA Rules not processed for user `12824152139841138f457dfe5c9758e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:17.140036 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-18ba7d38-d758-4998-83ef-8c206e58ebec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:17.157084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 825/1669] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:06:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:17.204231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a24ef1-9ebb-470d-8a15-60fb2a1dc931 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:17.223588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a24ef1-9ebb-470d-8a15-60fb2a1dc931 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:17.235692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a13a4-d4fa-4ed1-a3a5-bd0bf172e623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:17.236374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a13a4-d4fa-4ed1-a3a5-bd0bf172e623 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:17.236656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a13a4-d4fa-4ed1-a3a5-bd0bf172e623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:17.236933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a13a4-d4fa-4ed1-a3a5-bd0bf172e623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:17.258494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a13a4-d4fa-4ed1-a3a5-bd0bf172e623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:17.259956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a13a4-d4fa-4ed1-a3a5-bd0bf172e623 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:17.284885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 845/1670] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:17.623161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a6759-9b14-4e5c-9cf4-1e25e4ff1802 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:17.623647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a6759-9b14-4e5c-9cf4-1e25e4ff1802 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:17.623647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a6759-9b14-4e5c-9cf4-1e25e4ff1802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:17.680701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e28a6759-9b14-4e5c-9cf4-1e25e4ff1802 None None] MFA Rules not processed for user `35c7c83855084c188ab8a6dacb57a24b`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:17.682220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e28a6759-9b14-4e5c-9cf4-1e25e4ff1802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:17.734653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072baa-3058-4a71-87c0-b6dac0c3be9d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:17.735124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072baa-3058-4a71-87c0-b6dac0c3be9d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:17.735261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072baa-3058-4a71-87c0-b6dac0c3be9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:17.772752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 826/1671] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:17.774570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf8d22-04d4-43a8-a9db-6be71f44b431 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:17.775151 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf8d22-04d4-43a8-a9db-6be71f44b431 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:17.775779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf8d22-04d4-43a8-a9db-6be71f44b431 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:17.782210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 827/1672] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 13 09:06:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:06:17.784962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cae056-dee7-4ed7-a00e-8f5f61c307bc None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:17.785472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cae056-dee7-4ed7-a00e-8f5f61c307bc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:17.786239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cae056-dee7-4ed7-a00e-8f5f61c307bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:17.788878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6b072baa-3058-4a71-87c0-b6dac0c3be9d None None] MFA Rules not processed for user `d9bffb24cf26486a9a8947882b52ec98`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:17.790476 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6b072baa-3058-4a71-87c0-b6dac0c3be9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:17.810262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-42cae056-dee7-4ed7-a00e-8f5f61c307bc None None] MFA Rules not processed for user `12824152139841138f457dfe5c9758e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:17.819157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-42cae056-dee7-4ed7-a00e-8f5f61c307bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:17.819316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:17.825836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 828/1673] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:06:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:17.843572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cae056-dee7-4ed7-a00e-8f5f61c307bc None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:17.845285 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-42cae056-dee7-4ed7-a00e-8f5f61c307bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:17.861364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cae056-dee7-4ed7-a00e-8f5f61c307bc None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:17.874586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c67c21-eeee-4daf-9f0a-0dac929141a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:17.876383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c67c21-eeee-4daf-9f0a-0dac929141a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:17.876383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c67c21-eeee-4daf-9f0a-0dac929141a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:17.876383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c67c21-eeee-4daf-9f0a-0dac929141a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:17.898406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c67c21-eeee-4daf-9f0a-0dac929141a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:17.899752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c67c21-eeee-4daf-9f0a-0dac929141a8 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:17.915458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 846/1674] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:17.919656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 829/1675] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:06:17.928538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187d5c8-9ca0-4911-a0cf-a9fe13b69ae7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:17.928830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187d5c8-9ca0-4911-a0cf-a9fe13b69ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:17.929110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187d5c8-9ca0-4911-a0cf-a9fe13b69ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:17.965292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a187d5c8-9ca0-4911-a0cf-a9fe13b69ae7 None None] MFA Rules not processed for user `d9bffb24cf26486a9a8947882b52ec98`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:17.966944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a187d5c8-9ca0-4911-a0cf-a9fe13b69ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:17.976061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 847/1676] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:18.269739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c67c21-eeee-4daf-9f0a-0dac929141a8 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:18.271415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-05c67c21-eeee-4daf-9f0a-0dac929141a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:18.411892 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d371bf-abb9-4eff-89ff-ec31b8ff4bb0 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:18.412279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d371bf-abb9-4eff-89ff-ec31b8ff4bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:18.412683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d371bf-abb9-4eff-89ff-ec31b8ff4bb0 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:18.415269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 848/1677] 10.222.0.40 () {54 vars in 960 bytes} [Wed Jul 13 09:06:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 09:06:18.425205 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10537b06-49b0-438b-a5ad-c4f40d9aea56 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:18.425859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10537b06-49b0-438b-a5ad-c4f40d9aea56 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:18.426269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10537b06-49b0-438b-a5ad-c4f40d9aea56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:18.469909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-10537b06-49b0-438b-a5ad-c4f40d9aea56 None None] MFA Rules not processed for user `bca7c1164d3e4833a10ffc37cbc6f092`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:18.471445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-10537b06-49b0-438b-a5ad-c4f40d9aea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:18.528914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c67c21-eeee-4daf-9f0a-0dac929141a8 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:18.542259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e85e1-13bb-457f-bdd1-b49b820ea758 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:18.542985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e85e1-13bb-457f-bdd1-b49b820ea758 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:18.543273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e85e1-13bb-457f-bdd1-b49b820ea758 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:18.543563 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e85e1-13bb-457f-bdd1-b49b820ea758 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:18.563533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e85e1-13bb-457f-bdd1-b49b820ea758 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:18.565938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e85e1-13bb-457f-bdd1-b49b820ea758 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:18.605835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 830/1678] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:18.606817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e85e1-13bb-457f-bdd1-b49b820ea758 service nova] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:18.610140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fa0e85e1-13bb-457f-bdd1-b49b820ea758 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:18.647265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e85e1-13bb-457f-bdd1-b49b820ea758 service nova] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:18.658208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 849/1679] 10.222.0.40 () {58 vars in 1057 bytes} [Wed Jul 13 09:06:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:18.659522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:18.661264 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b5a47-aa48-45a2-b511-967bc46cc845 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:18.661794 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:18.662023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b5a47-aa48-45a2-b511-967bc46cc845 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:18.662177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:18.670920 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b5a47-aa48-45a2-b511-967bc46cc845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:18.671057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] PATH_INFO: `/v3/projects/537f3a6f839948638c78cd68ff9fcf24` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:18.671057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=537f3a6f839948638c78cd68ff9fcf24)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:18.671057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 09:06:18.671057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 13 09:06:18.671057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: Jul 13 09:06:18.671057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:18.716859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-506b5a47-aa48-45a2-b511-967bc46cc845 None None] MFA Rules not processed for user `fda6491fd5a144f687f86d9e1f7a7eed`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:18.718217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-506b5a47-aa48-45a2-b511-967bc46cc845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:18.838852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 850/1680] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:18 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:18.838852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b5a47-aa48-45a2-b511-967bc46cc845 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:18.859275 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:18.885021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '537f3a6f839948638c78cd68ff9fcf24', 'request_id': 'req-e3fb26b8-766e-4c90-a380-b27436558dcf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:18.891088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b5a47-aa48-45a2-b511-967bc46cc845 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:18.898993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '537f3a6f839948638c78cd68ff9fcf24', 'request_id': 'req-e3fb26b8-766e-4c90-a380-b27436558dcf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:18.969714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '537f3a6f839948638c78cd68ff9fcf24', 'request_id': 'req-e3fb26b8-766e-4c90-a380-b27436558dcf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:19.057202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] The token cache is being invalidate because project 537f3a6f839948638c78cd68ff9fcf24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:19.058331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:19.060943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:19.065812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 831/1681] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:18 2022] DELETE /identity/v3/projects/537f3a6f839948638c78cd68ff9fcf24 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:19.067043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:19.075456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:19.111926 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578e08e-9f54-4687-8a08-c9922f7311ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:19.113129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578e08e-9f54-4687-8a08-c9922f7311ec service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:19.113597 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578e08e-9f54-4687-8a08-c9922f7311ec service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:19.114012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578e08e-9f54-4687-8a08-c9922f7311ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:19.116676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8578e08e-9f54-4687-8a08-c9922f7311ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:19.130721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578e08e-9f54-4687-8a08-c9922f7311ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:19.132695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578e08e-9f54-4687-8a08-c9922f7311ec service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:19.145306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 851/1682] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:19.146724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578e08e-9f54-4687-8a08-c9922f7311ec service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:19.148376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8578e08e-9f54-4687-8a08-c9922f7311ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:19.267573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578e08e-9f54-4687-8a08-c9922f7311ec service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:19.273881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc39a64-0849-45cd-91c5-b180bef644c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:19.274812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc39a64-0849-45cd-91c5-b180bef644c6 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:19.275180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc39a64-0849-45cd-91c5-b180bef644c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:19.275525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc39a64-0849-45cd-91c5-b180bef644c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:19.278470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-abc39a64-0849-45cd-91c5-b180bef644c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:19.295869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb26b8-766e-4c90-a380-b27436558dcf admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:19.325407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:19.326927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:19.327224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:19.327565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] PATH_INFO: `/v3/users/4853ac8be8164693ae16919a59a93ada` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:19.331407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4853ac8be8164693ae16919a59a93ada)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:19.338051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:19.406962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc39a64-0849-45cd-91c5-b180bef644c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:19.409422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc39a64-0849-45cd-91c5-b180bef644c6 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:19.462486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 852/1683] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:19.464043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d0a36-6c30-4157-b4dc-d5ec53be1ee3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:19.464339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d0a36-6c30-4157-b4dc-d5ec53be1ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:19.464622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d0a36-6c30-4157-b4dc-d5ec53be1ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:19.507565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-970d0a36-6c30-4157-b4dc-d5ec53be1ee3 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:19.511949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-970d0a36-6c30-4157-b4dc-d5ec53be1ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:19.557124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 853/1684] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:19.558189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d0a36-6c30-4157-b4dc-d5ec53be1ee3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:19.559744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-970d0a36-6c30-4157-b4dc-d5ec53be1ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:19.563332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:19.607010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d0a36-6c30-4157-b4dc-d5ec53be1ee3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:19.638563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841aa056-0751-40c7-909b-3d0995829435 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:19.639485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aa056-0751-40c7-909b-3d0995829435 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:19.639769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aa056-0751-40c7-909b-3d0995829435 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:19.640099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aa056-0751-40c7-909b-3d0995829435 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:19.664114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841aa056-0751-40c7-909b-3d0995829435 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:19.665477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841aa056-0751-40c7-909b-3d0995829435 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:19.692677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4853ac8be8164693ae16919a59a93ada'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:19.694547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 854/1685] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:19.712635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e451-dd77-4314-a691-441b1dafbc2c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:19.712928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e451-dd77-4314-a691-441b1dafbc2c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:19.715302 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32e451-dd77-4314-a691-441b1dafbc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:19.724530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4853ac8be8164693ae16919a59a93ada'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:19.742451 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4853ac8be8164693ae16919a59a93ada'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:19.786575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7b32e451-dd77-4314-a691-441b1dafbc2c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:19.801393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7b32e451-dd77-4314-a691-441b1dafbc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:19.835799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4853ac8be8164693ae16919a59a93ada'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:19.838630 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-65922f8f-0bd8-4f63-a8e7-94cea2820f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4853ac8be8164693ae16919a59a93ada'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:19.845350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 855/1686] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:19.845350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32e451-dd77-4314-a691-441b1dafbc2c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:19.848903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 832/1687] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:19 2022] DELETE /identity/v3/users/4853ac8be8164693ae16919a59a93ada => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:19.853610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bb29d-8048-46bd-9dca-cd75bb14a195 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:19.853958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bb29d-8048-46bd-9dca-cd75bb14a195 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:19.854213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bb29d-8048-46bd-9dca-cd75bb14a195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:19.884894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32e451-dd77-4314-a691-441b1dafbc2c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:19.912885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-977bb29d-8048-46bd-9dca-cd75bb14a195 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:19.915314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-977bb29d-8048-46bd-9dca-cd75bb14a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:20.113999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde0256-d257-46a8-8f75-53f90296e9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:20.115751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde0256-d257-46a8-8f75-53f90296e9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:20.117198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde0256-d257-46a8-8f75-53f90296e9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:20.118475 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde0256-d257-46a8-8f75-53f90296e9d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:20.119366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde0256-d257-46a8-8f75-53f90296e9d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:06:20.119815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde0256-d257-46a8-8f75-53f90296e9d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:20.123654 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde0256-d257-46a8-8f75-53f90296e9d4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:20.160016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 856/1688] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:06:20.164437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde0256-d257-46a8-8f75-53f90296e9d4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:20.170772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3dde0256-d257-46a8-8f75-53f90296e9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:20.204478 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde0256-d257-46a8-8f75-53f90296e9d4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:20.207955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:20.215089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 833/1689] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:20.234358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bb29d-8048-46bd-9dca-cd75bb14a195 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:20.235945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-977bb29d-8048-46bd-9dca-cd75bb14a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:20.250158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1fd82-1754-4155-91a9-75d74a3a92f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:20.251548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1fd82-1754-4155-91a9-75d74a3a92f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:20.255208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1fd82-1754-4155-91a9-75d74a3a92f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:20.255682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1fd82-1754-4155-91a9-75d74a3a92f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:20.259101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1fd82-1754-4155-91a9-75d74a3a92f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:20.264234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1fd82-1754-4155-91a9-75d74a3a92f3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:20.383489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:20.442831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 857/1690] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:20 2022] POST /identity/v3/projects => generated 353 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:20.444404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1fd82-1754-4155-91a9-75d74a3a92f3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:20.445155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bb29d-8048-46bd-9dca-cd75bb14a195 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:20.594726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9374af-efb7-4610-ad4d-b14568b6507b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:20.595383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9374af-efb7-4610-ad4d-b14568b6507b service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:20.595661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9374af-efb7-4610-ad4d-b14568b6507b service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:20.595931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9374af-efb7-4610-ad4d-b14568b6507b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:20.624792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1fd82-1754-4155-91a9-75d74a3a92f3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:20.633227 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcd89b-0493-406e-91fb-b9879bbb2741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:20.634356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcd89b-0493-406e-91fb-b9879bbb2741 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:20.634690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcd89b-0493-406e-91fb-b9879bbb2741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:20.635022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcd89b-0493-406e-91fb-b9879bbb2741 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:20.639519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcd89b-0493-406e-91fb-b9879bbb2741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:20.643711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9374af-efb7-4610-ad4d-b14568b6507b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:20.643815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcd89b-0493-406e-91fb-b9879bbb2741 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:20.643969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9374af-efb7-4610-ad4d-b14568b6507b service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:20.688933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 834/1691] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:20.689912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9374af-efb7-4610-ad4d-b14568b6507b service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:20.695344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ea9374af-efb7-4610-ad4d-b14568b6507b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:20.732092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9374af-efb7-4610-ad4d-b14568b6507b service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:20.746166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:20.746166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:20.746166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:20.748915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] PATH_INFO: `/v3/users/5b4fb5ca6bd04466bc430c891cc40742` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:20.750491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4fb5ca6bd04466bc430c891cc40742)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:20.753198 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:20.777910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 858/1692] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:20 2022] POST /identity/v3/users => generated 277 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:20.779167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcd89b-0493-406e-91fb-b9879bbb2741 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:20.811992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcd89b-0493-406e-91fb-b9879bbb2741 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:20.826104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96c15e-d804-4dee-8a1b-d8c7462d3955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:20.841595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96c15e-d804-4dee-8a1b-d8c7462d3955 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:20.843872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96c15e-d804-4dee-8a1b-d8c7462d3955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:20.844725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96c15e-d804-4dee-8a1b-d8c7462d3955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:20.851433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96c15e-d804-4dee-8a1b-d8c7462d3955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:20.855667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96c15e-d804-4dee-8a1b-d8c7462d3955 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:20.888580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 859/1693] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:20.889965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96c15e-d804-4dee-8a1b-d8c7462d3955 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:20.904347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96c15e-d804-4dee-8a1b-d8c7462d3955 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:20.913312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d33473-afd1-4d8e-8a1e-f6aa4c506963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:20.914544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d33473-afd1-4d8e-8a1e-f6aa4c506963 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:20.915043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d33473-afd1-4d8e-8a1e-f6aa4c506963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:20.915542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d33473-afd1-4d8e-8a1e-f6aa4c506963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:20.916535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d33473-afd1-4d8e-8a1e-f6aa4c506963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:20.918522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d33473-afd1-4d8e-8a1e-f6aa4c506963 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:20.934610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 860/1694] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:20.938940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d33473-afd1-4d8e-8a1e-f6aa4c506963 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:20.959418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d33473-afd1-4d8e-8a1e-f6aa4c506963 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:20.966025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921664c7-04ec-4525-9d88-b412e3899ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:20.967337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921664c7-04ec-4525-9d88-b412e3899ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:20.967815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921664c7-04ec-4525-9d88-b412e3899ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:20.970307 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921664c7-04ec-4525-9d88-b412e3899ad8 admin admin] PATH_INFO: `/v3/projects/27a5e75617e749908a9fdf29f91ec3e2/users/41311259448942d896d3c85715a67d46/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:21.009225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921664c7-04ec-4525-9d88-b412e3899ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a5e75617e749908a9fdf29f91ec3e2, user_id=41311259448942d896d3c85715a67d46, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:21.014982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921664c7-04ec-4525-9d88-b412e3899ad8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:21.101774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4fb5ca6bd04466bc430c891cc40742'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:21.118870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4fb5ca6bd04466bc430c891cc40742'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:21.136777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4fb5ca6bd04466bc430c891cc40742'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:21.152419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 861/1695] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:20 2022] PUT /identity/v3/projects/27a5e75617e749908a9fdf29f91ec3e2/users/41311259448942d896d3c85715a67d46/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:21.163123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de30b61-d1bc-402c-aa6d-f72330fa040f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:21.163747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de30b61-d1bc-402c-aa6d-f72330fa040f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:21.164249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de30b61-d1bc-402c-aa6d-f72330fa040f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:21.225185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7de30b61-d1bc-402c-aa6d-f72330fa040f None None] MFA Rules not processed for user `41311259448942d896d3c85715a67d46`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:21.226686 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7de30b61-d1bc-402c-aa6d-f72330fa040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:21.253684 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4fb5ca6bd04466bc430c891cc40742'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:21.254453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-88d30781-81c7-4ceb-8a88-27de2218af50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4fb5ca6bd04466bc430c891cc40742'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:21.269204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 835/1696] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:20 2022] DELETE /identity/v3/users/5b4fb5ca6bd04466bc430c891cc40742 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:21.273046 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db277315-a724-4a6d-a317-54dc56f82a61 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:21.274683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db277315-a724-4a6d-a317-54dc56f82a61 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:21.274936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db277315-a724-4a6d-a317-54dc56f82a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:21.369497 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-db277315-a724-4a6d-a317-54dc56f82a61 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:21.374468 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:21.378678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-db277315-a724-4a6d-a317-54dc56f82a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:21.489627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 862/1697] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:21 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:21.495438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de30b61-d1bc-402c-aa6d-f72330fa040f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:21.497200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7de30b61-d1bc-402c-aa6d-f72330fa040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:21.636179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 836/1698] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:21.675457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db277315-a724-4a6d-a317-54dc56f82a61 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:21.677735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-db277315-a724-4a6d-a317-54dc56f82a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:21.760216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de30b61-d1bc-402c-aa6d-f72330fa040f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:21.774871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216283a-311f-467d-b827-afcdbfddf23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:21.775577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216283a-311f-467d-b827-afcdbfddf23f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:21.775867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216283a-311f-467d-b827-afcdbfddf23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:21.776144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216283a-311f-467d-b827-afcdbfddf23f admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:21.776860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216283a-311f-467d-b827-afcdbfddf23f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:21.778832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216283a-311f-467d-b827-afcdbfddf23f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:21.875025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db277315-a724-4a6d-a317-54dc56f82a61 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:21.896899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75111504-7cbb-4cdb-acf4-25956a6f188e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:21.897286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75111504-7cbb-4cdb-acf4-25956a6f188e service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:21.897580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75111504-7cbb-4cdb-acf4-25956a6f188e service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:21.897917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75111504-7cbb-4cdb-acf4-25956a6f188e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:21.917907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75111504-7cbb-4cdb-acf4-25956a6f188e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:21.922653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75111504-7cbb-4cdb-acf4-25956a6f188e service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:21.954869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 837/1699] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:21.960271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa00ba-c252-4f12-bd20-661e9bc09f65 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:21.960271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa00ba-c252-4f12-bd20-661e9bc09f65 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:21.960271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa00ba-c252-4f12-bd20-661e9bc09f65 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:21.962563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 863/1700] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:21 2022] POST /identity/v3/projects => generated 357 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:21.964560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 838/1701] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 09:06:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:06:21.966655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216283a-311f-467d-b827-afcdbfddf23f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:22.027090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44ca6c-4685-4dc5-80d9-d59fd2ea5fee None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:22.027443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44ca6c-4685-4dc5-80d9-d59fd2ea5fee None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:22.027656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44ca6c-4685-4dc5-80d9-d59fd2ea5fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:22.052353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fe44ca6c-4685-4dc5-80d9-d59fd2ea5fee None None] MFA Rules not processed for user `bca7c1164d3e4833a10ffc37cbc6f092`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:22.054460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fe44ca6c-4685-4dc5-80d9-d59fd2ea5fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:22.113523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216283a-311f-467d-b827-afcdbfddf23f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:22.125433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55118867-864c-484b-bc8c-ecf9390dab1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:22.126487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55118867-864c-484b-bc8c-ecf9390dab1b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:22.127029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55118867-864c-484b-bc8c-ecf9390dab1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:22.127506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55118867-864c-484b-bc8c-ecf9390dab1b admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:22.131569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55118867-864c-484b-bc8c-ecf9390dab1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:22.135403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55118867-864c-484b-bc8c-ecf9390dab1b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:22.219119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 839/1702] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 09:06:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:22.220921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc51ae-08a5-4342-8cdc-b865511ec9d6 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:22.221492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc51ae-08a5-4342-8cdc-b865511ec9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:22.221785 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc51ae-08a5-4342-8cdc-b865511ec9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:22.246480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 864/1703] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:21 2022] POST /identity/v3/users => generated 285 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:22.247428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55118867-864c-484b-bc8c-ecf9390dab1b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:22.251516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-cfdc51ae-08a5-4342-8cdc-b865511ec9d6 None None] MFA Rules not processed for user `d9bffb24cf26486a9a8947882b52ec98`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:22.254781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-55118867-864c-484b-bc8c-ecf9390dab1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:22.262216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cfdc51ae-08a5-4342-8cdc-b865511ec9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:22.283886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55118867-864c-484b-bc8c-ecf9390dab1b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:22.302989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:22.303931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:22.304216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:22.305108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] PATH_INFO: `/v3/projects/c1bcbd0180e94d7eb8a4e1ed1eb48931` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:22.307249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1bcbd0180e94d7eb8a4e1ed1eb48931)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:22.310154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:22.437848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 840/1704] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:22.438837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc51ae-08a5-4342-8cdc-b865511ec9d6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:22.486289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc51ae-08a5-4342-8cdc-b865511ec9d6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:22.496402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e46331-05b4-475e-acbe-e41741c379e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:22.496402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e46331-05b4-475e-acbe-e41741c379e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:22.496402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e46331-05b4-475e-acbe-e41741c379e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:22.496402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e46331-05b4-475e-acbe-e41741c379e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:22.497095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e46331-05b4-475e-acbe-e41741c379e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:22.499074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e46331-05b4-475e-acbe-e41741c379e1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:22.511319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 841/1705] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:22.512269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e46331-05b4-475e-acbe-e41741c379e1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:22.515168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-17e46331-05b4-475e-acbe-e41741c379e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:22.536781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e46331-05b4-475e-acbe-e41741c379e1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:22.564395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ec9a8-bdc4-4a56-8899-f5d213402b29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:22.564395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ec9a8-bdc4-4a56-8899-f5d213402b29 service placement] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:22.564395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ec9a8-bdc4-4a56-8899-f5d213402b29 service placement] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:22.564395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ec9a8-bdc4-4a56-8899-f5d213402b29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:22.576233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ec9a8-bdc4-4a56-8899-f5d213402b29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:22.577847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ec9a8-bdc4-4a56-8899-f5d213402b29 service placement] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:22.614085 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 842/1706] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:22.617635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1bcbd0180e94d7eb8a4e1ed1eb48931', 'request_id': 'req-6fb3b116-2e64-405d-a858-25856de21d84'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:22.619027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ec9a8-bdc4-4a56-8899-f5d213402b29 service placement] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:22.631212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1bcbd0180e94d7eb8a4e1ed1eb48931', 'request_id': 'req-6fb3b116-2e64-405d-a858-25856de21d84'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:22.645860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ec9a8-bdc4-4a56-8899-f5d213402b29 service placement] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:22.649859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1bcbd0180e94d7eb8a4e1ed1eb48931', 'request_id': 'req-6fb3b116-2e64-405d-a858-25856de21d84'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:22.720053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3e992-4f75-4af6-b1d4-ce20228502bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:22.722067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3e992-4f75-4af6-b1d4-ce20228502bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:22.722067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3e992-4f75-4af6-b1d4-ce20228502bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:22.722067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3e992-4f75-4af6-b1d4-ce20228502bd admin admin] PATH_INFO: `/v3/projects/68960387660f4d29abe165d68f77b90f/users/342cd72c66ed477580ebe746b82d5193/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:22.735244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] The token cache is being invalidate because project c1bcbd0180e94d7eb8a4e1ed1eb48931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:22.738734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:22.738734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6fb3b116-2e64-405d-a858-25856de21d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:22.743617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 865/1707] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:22 2022] DELETE /identity/v3/projects/c1bcbd0180e94d7eb8a4e1ed1eb48931 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:22.748704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2038a-146b-490d-a818-19e067ddd0d9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:22.748926 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2038a-146b-490d-a818-19e067ddd0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:22.749262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2038a-146b-490d-a818-19e067ddd0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:22.772757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3e992-4f75-4af6-b1d4-ce20228502bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=68960387660f4d29abe165d68f77b90f, user_id=342cd72c66ed477580ebe746b82d5193, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:22.780879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3e992-4f75-4af6-b1d4-ce20228502bd admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:22.791160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5ad2038a-146b-490d-a818-19e067ddd0d9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:22.792502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5ad2038a-146b-490d-a818-19e067ddd0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:22.804796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 866/1708] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:22.813201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a5a0f-cb1c-4ba5-bd24-3731d279fc10 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:22.814591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a5a0f-cb1c-4ba5-bd24-3731d279fc10 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:22.814591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a5a0f-cb1c-4ba5-bd24-3731d279fc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:22.861103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 843/1709] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:22 2022] PUT /identity/v3/projects/68960387660f4d29abe165d68f77b90f/users/342cd72c66ed477580ebe746b82d5193/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:22.867614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3e992-4f75-4af6-b1d4-ce20228502bd admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:22.870702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dee3e992-4f75-4af6-b1d4-ce20228502bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:22.909864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-572a5a0f-cb1c-4ba5-bd24-3731d279fc10 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:22.910998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-572a5a0f-cb1c-4ba5-bd24-3731d279fc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:23.013835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:23.104829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3e992-4f75-4af6-b1d4-ce20228502bd admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:23.114980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e5a21-f42f-4e56-ad89-5bc488d95c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:23.115444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e5a21-f42f-4e56-ad89-5bc488d95c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:23.115732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e5a21-f42f-4e56-ad89-5bc488d95c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:23.117047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e5a21-f42f-4e56-ad89-5bc488d95c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:23.117793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e5a21-f42f-4e56-ad89-5bc488d95c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:23.126649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e5a21-f42f-4e56-ad89-5bc488d95c4a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:23.146677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 844/1710] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:23.151485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e5a21-f42f-4e56-ad89-5bc488d95c4a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:23.168980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e5a21-f42f-4e56-ad89-5bc488d95c4a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:23.170704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:23.179437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa135b-8889-4fb4-8c9c-0e487d6e2f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:23.180415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa135b-8889-4fb4-8c9c-0e487d6e2f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:23.180867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa135b-8889-4fb4-8c9c-0e487d6e2f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:23.181169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa135b-8889-4fb4-8c9c-0e487d6e2f8b admin admin] PATH_INFO: `/v3/projects/68960387660f4d29abe165d68f77b90f/users/342cd72c66ed477580ebe746b82d5193/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:23.185732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa135b-8889-4fb4-8c9c-0e487d6e2f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=68960387660f4d29abe165d68f77b90f, user_id=342cd72c66ed477580ebe746b82d5193, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:23.196541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa135b-8889-4fb4-8c9c-0e487d6e2f8b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:23.198817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 867/1711] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:23.200117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a5a0f-cb1c-4ba5-bd24-3731d279fc10 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:23.218432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 845/1712] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:23 2022] PUT /identity/v3/projects/68960387660f4d29abe165d68f77b90f/users/342cd72c66ed477580ebe746b82d5193/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:23.230535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30418ee7-41d0-46da-9290-8d497d2f1bc1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:23.230984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30418ee7-41d0-46da-9290-8d497d2f1bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:23.231434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30418ee7-41d0-46da-9290-8d497d2f1bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:23.236251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a5a0f-cb1c-4ba5-bd24-3731d279fc10 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:23.354022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-30418ee7-41d0-46da-9290-8d497d2f1bc1 None None] MFA Rules not processed for user `342cd72c66ed477580ebe746b82d5193`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:23.358350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-30418ee7-41d0-46da-9290-8d497d2f1bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:23.445344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2768848-956e-4707-8017-ab89396a0c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:23.447179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2768848-956e-4707-8017-ab89396a0c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:23.447656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2768848-956e-4707-8017-ab89396a0c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:23.448210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2768848-956e-4707-8017-ab89396a0c45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:23.450688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2768848-956e-4707-8017-ab89396a0c45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:06:23.452610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2768848-956e-4707-8017-ab89396a0c45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:23.458503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2768848-956e-4707-8017-ab89396a0c45 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:23.502830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 868/1713] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:06:23.504985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2768848-956e-4707-8017-ab89396a0c45 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:23.543072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2768848-956e-4707-8017-ab89396a0c45 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:23.558173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22b1ab-eb72-450a-97e6-3e51ca330c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:23.559338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22b1ab-eb72-450a-97e6-3e51ca330c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:23.561978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22b1ab-eb72-450a-97e6-3e51ca330c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:23.562314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22b1ab-eb72-450a-97e6-3e51ca330c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:23.563104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22b1ab-eb72-450a-97e6-3e51ca330c17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:23.570174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22b1ab-eb72-450a-97e6-3e51ca330c17 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:23.673268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 846/1714] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:23 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:23.688439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30418ee7-41d0-46da-9290-8d497d2f1bc1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:23.689598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-30418ee7-41d0-46da-9290-8d497d2f1bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:23.754297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 869/1715] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:23 2022] POST /identity/v3/projects => generated 387 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:23.756902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22b1ab-eb72-450a-97e6-3e51ca330c17 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:23.890186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30418ee7-41d0-46da-9290-8d497d2f1bc1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:23.897909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22b1ab-eb72-450a-97e6-3e51ca330c17 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:23.905985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd7db4-d6d6-403a-bcc5-94f3cfbe4588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:23.907441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd7db4-d6d6-403a-bcc5-94f3cfbe4588 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:23.907552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd7db4-d6d6-403a-bcc5-94f3cfbe4588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:23.907821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd7db4-d6d6-403a-bcc5-94f3cfbe4588 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:23.909335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd7db4-d6d6-403a-bcc5-94f3cfbe4588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:23.911452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd7db4-d6d6-403a-bcc5-94f3cfbe4588 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:24.013343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a93af-5c46-4596-94a7-7344d6747413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:24.014013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a93af-5c46-4596-94a7-7344d6747413 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:24.014623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a93af-5c46-4596-94a7-7344d6747413 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:24.014623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a93af-5c46-4596-94a7-7344d6747413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:24.041193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 870/1716] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:23 2022] POST /identity/v3/users => generated 294 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:24.042396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd7db4-d6d6-403a-bcc5-94f3cfbe4588 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:24.054171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd7db4-d6d6-403a-bcc5-94f3cfbe4588 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:24.060318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2826dd5-80c2-4bb6-b099-6d4bea5dc840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:24.067074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2826dd5-80c2-4bb6-b099-6d4bea5dc840 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:24.067365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2826dd5-80c2-4bb6-b099-6d4bea5dc840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:24.067717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2826dd5-80c2-4bb6-b099-6d4bea5dc840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:24.068562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2826dd5-80c2-4bb6-b099-6d4bea5dc840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:24.070748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2826dd5-80c2-4bb6-b099-6d4bea5dc840 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:24.091941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 871/1717] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:24.093293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2826dd5-80c2-4bb6-b099-6d4bea5dc840 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:24.160657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2826dd5-80c2-4bb6-b099-6d4bea5dc840 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:24.187057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ce5f9-85b2-49f1-9c1b-50c7b63e9a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:24.187760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ce5f9-85b2-49f1-9c1b-50c7b63e9a72 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:24.194812 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a93af-5c46-4596-94a7-7344d6747413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:24.194950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ce5f9-85b2-49f1-9c1b-50c7b63e9a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:24.195100 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ce5f9-85b2-49f1-9c1b-50c7b63e9a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:24.196260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a93af-5c46-4596-94a7-7344d6747413 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:24.199266 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ce5f9-85b2-49f1-9c1b-50c7b63e9a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:24.203188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ce5f9-85b2-49f1-9c1b-50c7b63e9a72 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:24.218106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 847/1718] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:23 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:24.228419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a93af-5c46-4596-94a7-7344d6747413 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:24.237359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-476a93af-5c46-4596-94a7-7344d6747413 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:24.263451 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a93af-5c46-4596-94a7-7344d6747413 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:24.272103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:24.273196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:24.273611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:24.274211 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] PATH_INFO: `/v3/projects/ac97aef9caa54a8eb844b6c075c5ff48` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:24.275769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac97aef9caa54a8eb844b6c075c5ff48)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:24.278499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:24.287448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 872/1719] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:24.291063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ce5f9-85b2-49f1-9c1b-50c7b63e9a72 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:24.313203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ce5f9-85b2-49f1-9c1b-50c7b63e9a72 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:24.377085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a3e7-6d94-4251-aac2-b8f4aaca3148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:24.378866 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a3e7-6d94-4251-aac2-b8f4aaca3148 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:24.386708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a3e7-6d94-4251-aac2-b8f4aaca3148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:24.394390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a3e7-6d94-4251-aac2-b8f4aaca3148 admin admin] PATH_INFO: `/v3/projects/30ef5f06036840549f823270953ec539/users/d20b89a832d844dcac69081a9ab0222b/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:24.454341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac97aef9caa54a8eb844b6c075c5ff48', 'request_id': 'req-8e050312-87e8-4f6d-95bc-9a8d61eb9734'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:24.454585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8a3e7-6d94-4251-aac2-b8f4aaca3148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ef5f06036840549f823270953ec539, user_id=d20b89a832d844dcac69081a9ab0222b, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:24.463417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8a3e7-6d94-4251-aac2-b8f4aaca3148 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:24.475327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac97aef9caa54a8eb844b6c075c5ff48', 'request_id': 'req-8e050312-87e8-4f6d-95bc-9a8d61eb9734'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:24.571891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac97aef9caa54a8eb844b6c075c5ff48', 'request_id': 'req-8e050312-87e8-4f6d-95bc-9a8d61eb9734'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:24.578657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 873/1720] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:24 2022] PUT /identity/v3/projects/30ef5f06036840549f823270953ec539/users/d20b89a832d844dcac69081a9ab0222b/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:24.590134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a3e7-6d94-4251-aac2-b8f4aaca3148 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:24.602730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] The token cache is being invalidate because project ac97aef9caa54a8eb844b6c075c5ff48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:24.605425 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:24.610713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8e050312-87e8-4f6d-95bc-9a8d61eb9734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:24.617869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 848/1721] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:24 2022] DELETE /identity/v3/projects/ac97aef9caa54a8eb844b6c075c5ff48 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:24.624367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b200cc-4fea-4fb2-8319-e29791cfac9f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:24.626622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b200cc-4fea-4fb2-8319-e29791cfac9f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:24.629560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b200cc-4fea-4fb2-8319-e29791cfac9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:24.696146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-36b200cc-4fea-4fb2-8319-e29791cfac9f None None] MFA Rules not processed for user `d20b89a832d844dcac69081a9ab0222b`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:24.700437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-36b200cc-4fea-4fb2-8319-e29791cfac9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:24.792084 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a3e7-6d94-4251-aac2-b8f4aaca3148 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:24.817731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92165b-2676-4cde-8bba-a2552cbf9f39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:24.822199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92165b-2676-4cde-8bba-a2552cbf9f39 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:24.822575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92165b-2676-4cde-8bba-a2552cbf9f39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:24.822934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92165b-2676-4cde-8bba-a2552cbf9f39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:24.831698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6c92165b-2676-4cde-8bba-a2552cbf9f39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:24.872896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 849/1722] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:24.873862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b200cc-4fea-4fb2-8319-e29791cfac9f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:24.886549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-36b200cc-4fea-4fb2-8319-e29791cfac9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:24.995227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b200cc-4fea-4fb2-8319-e29791cfac9f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:25.000445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:25.002315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:25.002644 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:25.002928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] PATH_INFO: `/v3/users/27f522688c2a4d558080ee7c5786ebfe` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:25.004372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27f522688c2a4d558080ee7c5786ebfe)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:25.006972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:25.062074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92165b-2676-4cde-8bba-a2552cbf9f39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:25.063541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92165b-2676-4cde-8bba-a2552cbf9f39 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:25.103891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 874/1723] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:24 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:25.105902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92165b-2676-4cde-8bba-a2552cbf9f39 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:25.109959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6c92165b-2676-4cde-8bba-a2552cbf9f39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:25.176701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92165b-2676-4cde-8bba-a2552cbf9f39 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:25.200339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5822f1d-4b41-4576-8745-f946b5e41fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:25.203100 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5822f1d-4b41-4576-8745-f946b5e41fcb admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:25.203430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5822f1d-4b41-4576-8745-f946b5e41fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:25.203718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5822f1d-4b41-4576-8745-f946b5e41fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:25.204491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5822f1d-4b41-4576-8745-f946b5e41fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:25.209292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5822f1d-4b41-4576-8745-f946b5e41fcb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:25.212067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:25.322376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:25.355104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27f522688c2a4d558080ee7c5786ebfe'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:25.371444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27f522688c2a4d558080ee7c5786ebfe'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:25.392007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 875/1724] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:25 2022] POST /identity/v3/projects => generated 387 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:25.407933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5822f1d-4b41-4576-8745-f946b5e41fcb admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:25.425900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27f522688c2a4d558080ee7c5786ebfe'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:25.501561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5822f1d-4b41-4576-8745-f946b5e41fcb admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:25.507297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bbf2e-d330-4f36-9636-eaacdd509223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:25.508043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bbf2e-d330-4f36-9636-eaacdd509223 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:25.508461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bbf2e-d330-4f36-9636-eaacdd509223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:25.508744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bbf2e-d330-4f36-9636-eaacdd509223 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:25.509640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678bbf2e-d330-4f36-9636-eaacdd509223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:25.512277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678bbf2e-d330-4f36-9636-eaacdd509223 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:25.552781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27f522688c2a4d558080ee7c5786ebfe'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:25.552781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27f522688c2a4d558080ee7c5786ebfe'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:25.576332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 850/1725] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:24 2022] DELETE /identity/v3/users/27f522688c2a4d558080ee7c5786ebfe => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:25.667888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 876/1726] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:25 2022] POST /identity/v3/users => generated 294 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:25.671717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:25.674742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:25.790635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf615e-0d0a-4c4c-b9de-6d0ac7326440 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:25.791255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf615e-0d0a-4c4c-b9de-6d0ac7326440 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:25.792321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf615e-0d0a-4c4c-b9de-6d0ac7326440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:25.895260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273acdf5-f930-4623-9e94-b482bbb34865 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:25.908135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc8f2d-5b87-49c5-99d5-0e4ffbeb4dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:25.908980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc8f2d-5b87-49c5-99d5-0e4ffbeb4dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:25.909155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc8f2d-5b87-49c5-99d5-0e4ffbeb4dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:25.909409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc8f2d-5b87-49c5-99d5-0e4ffbeb4dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:25.910182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc8f2d-5b87-49c5-99d5-0e4ffbeb4dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:25.918774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc8f2d-5b87-49c5-99d5-0e4ffbeb4dd4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:25.938826 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 851/1727] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:25.945185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc8f2d-5b87-49c5-99d5-0e4ffbeb4dd4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:25.953847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1bdc8f2d-5b87-49c5-99d5-0e4ffbeb4dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:25.993638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc8f2d-5b87-49c5-99d5-0e4ffbeb4dd4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:26.005425 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:26.016805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:26.016805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:26.016805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] PATH_INFO: `/v3/users/cef9f25657744847aac4ef86f34b12ed` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:26.017200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fecf615e-0d0a-4c4c-b9de-6d0ac7326440 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:26.017427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cef9f25657744847aac4ef86f34b12ed)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:26.017769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fecf615e-0d0a-4c4c-b9de-6d0ac7326440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:26.017940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:26.084175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:26.160504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 877/1728] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:06:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:26.161372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf615e-0d0a-4c4c-b9de-6d0ac7326440 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:26.178711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf615e-0d0a-4c4c-b9de-6d0ac7326440 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:26.191357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4ed0e-78af-4c43-8cf4-2f71c024235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:26.194844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ed0e-78af-4c43-8cf4-2f71c024235a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:26.194844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ed0e-78af-4c43-8cf4-2f71c024235a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:26.194844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4ed0e-78af-4c43-8cf4-2f71c024235a admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:26.194844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4ed0e-78af-4c43-8cf4-2f71c024235a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:26.195154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4ed0e-78af-4c43-8cf4-2f71c024235a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:26.212079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 878/1729] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:26.229337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4ed0e-78af-4c43-8cf4-2f71c024235a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:26.243824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cef9f25657744847aac4ef86f34b12ed'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:26.255216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4ed0e-78af-4c43-8cf4-2f71c024235a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:26.264236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cef9f25657744847aac4ef86f34b12ed'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:26.272699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601db8e-03bc-40b1-b6b2-f18ddeea5728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:26.274082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601db8e-03bc-40b1-b6b2-f18ddeea5728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:26.274082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601db8e-03bc-40b1-b6b2-f18ddeea5728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:26.276968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601db8e-03bc-40b1-b6b2-f18ddeea5728 admin admin] PATH_INFO: `/v3/projects/f5cb192da07e410cbd80f0a987265b74/users/6a5f4b270ec24099904f5e626a9a0aee/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:26.295965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cef9f25657744847aac4ef86f34b12ed'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:26.312899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601db8e-03bc-40b1-b6b2-f18ddeea5728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5cb192da07e410cbd80f0a987265b74, user_id=6a5f4b270ec24099904f5e626a9a0aee, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:26.321630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601db8e-03bc-40b1-b6b2-f18ddeea5728 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:26.372120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cef9f25657744847aac4ef86f34b12ed'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:26.379255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-476c4184-3e4b-48c0-90df-43d7846bd40f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cef9f25657744847aac4ef86f34b12ed'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:26.388237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 852/1730] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:25 2022] DELETE /identity/v3/users/cef9f25657744847aac4ef86f34b12ed => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:26.416217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 879/1731] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:26 2022] PUT /identity/v3/projects/f5cb192da07e410cbd80f0a987265b74/users/6a5f4b270ec24099904f5e626a9a0aee/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:26.418294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7659e-432b-4011-8234-429fb8e5a401 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:26.418582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7659e-432b-4011-8234-429fb8e5a401 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:26.418861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7659e-432b-4011-8234-429fb8e5a401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:26.503154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d9c7659e-432b-4011-8234-429fb8e5a401 None None] MFA Rules not processed for user `6a5f4b270ec24099904f5e626a9a0aee`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:26.504616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d9c7659e-432b-4011-8234-429fb8e5a401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:26.577397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601db8e-03bc-40b1-b6b2-f18ddeea5728 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:26.579598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5601db8e-03bc-40b1-b6b2-f18ddeea5728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:26.721846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 853/1732] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:26.723211 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211c9fb-c29c-4f5d-8d63-7cf800ab1475 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:26.723498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211c9fb-c29c-4f5d-8d63-7cf800ab1475 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:26.723779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211c9fb-c29c-4f5d-8d63-7cf800ab1475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:26.769659 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b211c9fb-c29c-4f5d-8d63-7cf800ab1475 None None] MFA Rules not processed for user `35c7c83855084c188ab8a6dacb57a24b`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:26.772253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b211c9fb-c29c-4f5d-8d63-7cf800ab1475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:26.872565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601db8e-03bc-40b1-b6b2-f18ddeea5728 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:26.886462 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 854/1733] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:26.886462 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211c9fb-c29c-4f5d-8d63-7cf800ab1475 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:26.888298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b211c9fb-c29c-4f5d-8d63-7cf800ab1475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:26.893631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5241aea-7ddc-4f34-9f0d-371c2832ace6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:26.894377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5241aea-7ddc-4f34-9f0d-371c2832ace6 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:26.894672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5241aea-7ddc-4f34-9f0d-371c2832ace6 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:26.894955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5241aea-7ddc-4f34-9f0d-371c2832ace6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:26.897833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a5241aea-7ddc-4f34-9f0d-371c2832ace6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:27.018552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211c9fb-c29c-4f5d-8d63-7cf800ab1475 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:27.044068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15804aeb-f140-441d-8625-d6e66fdd5cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:27.044854 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804aeb-f140-441d-8625-d6e66fdd5cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:27.045208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804aeb-f140-441d-8625-d6e66fdd5cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:27.045551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804aeb-f140-441d-8625-d6e66fdd5cfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:27.046358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15804aeb-f140-441d-8625-d6e66fdd5cfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:27.048599 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15804aeb-f140-441d-8625-d6e66fdd5cfd admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:27.164944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5241aea-7ddc-4f34-9f0d-371c2832ace6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:27.171658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5241aea-7ddc-4f34-9f0d-371c2832ace6 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:27.191918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 855/1734] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:26 2022] POST /identity/v3/projects => generated 385 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:27.196100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15804aeb-f140-441d-8625-d6e66fdd5cfd admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:27.212548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 880/1735] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:06:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:06:27.296721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15804aeb-f140-441d-8625-d6e66fdd5cfd admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:27.305374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4374f-181d-4e24-80b1-bdbb41ef50e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:27.306268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4374f-181d-4e24-80b1-bdbb41ef50e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:27.306613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4374f-181d-4e24-80b1-bdbb41ef50e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:27.306896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4374f-181d-4e24-80b1-bdbb41ef50e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:27.324707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4374f-181d-4e24-80b1-bdbb41ef50e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:27.327740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4374f-181d-4e24-80b1-bdbb41ef50e9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:27.454488 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 856/1736] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:27 2022] POST /identity/v3/users => generated 299 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:27.457374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5241aea-7ddc-4f34-9f0d-371c2832ace6 service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:27.479352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5241aea-7ddc-4f34-9f0d-371c2832ace6 service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:27.497546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d4a5c-fd6b-4bd6-a4e0-3cf4c00ea075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:27.503442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d4a5c-fd6b-4bd6-a4e0-3cf4c00ea075 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:27.504254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d4a5c-fd6b-4bd6-a4e0-3cf4c00ea075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:27.504733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d4a5c-fd6b-4bd6-a4e0-3cf4c00ea075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:27.510145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d4a5c-fd6b-4bd6-a4e0-3cf4c00ea075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:27.517655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d4a5c-fd6b-4bd6-a4e0-3cf4c00ea075 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:27.591266 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 881/1737] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:27.600071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:27.600932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4374f-181d-4e24-80b1-bdbb41ef50e9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:27.625753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4374f-181d-4e24-80b1-bdbb41ef50e9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:27.666431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8739f-d906-4dce-bf46-b0e432832ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:27.671998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8739f-d906-4dce-bf46-b0e432832ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:27.672317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8739f-d906-4dce-bf46-b0e432832ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:27.682545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8739f-d906-4dce-bf46-b0e432832ff7 admin admin] PATH_INFO: `/v3/projects/3248d63ed2b447aea806395f16188f14/users/0df9bb1a720741768e7bfc1a7b30a8a2/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:27.709015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:27.736140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8739f-d906-4dce-bf46-b0e432832ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3248d63ed2b447aea806395f16188f14, user_id=0df9bb1a720741768e7bfc1a7b30a8a2, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:27.745593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8739f-d906-4dce-bf46-b0e432832ff7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:27.889361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 857/1738] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:27 2022] PUT /identity/v3/projects/3248d63ed2b447aea806395f16188f14/users/0df9bb1a720741768e7bfc1a7b30a8a2/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:27.898362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d4a5c-fd6b-4bd6-a4e0-3cf4c00ea075 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:27.920836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d4a5c-fd6b-4bd6-a4e0-3cf4c00ea075 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:28.056079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976ae16-c55c-42e5-9a87-4439d443f97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:28.060158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976ae16-c55c-42e5-9a87-4439d443f97a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:28.060440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976ae16-c55c-42e5-9a87-4439d443f97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:28.065772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976ae16-c55c-42e5-9a87-4439d443f97a admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:28.072809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976ae16-c55c-42e5-9a87-4439d443f97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:28.083397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976ae16-c55c-42e5-9a87-4439d443f97a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:28.147788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 882/1739] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:06:28.152724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8739f-d906-4dce-bf46-b0e432832ff7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:28.168314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8739f-d906-4dce-bf46-b0e432832ff7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:28.190654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb600d-e470-4b53-b115-619cd1072043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:28.194280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb600d-e470-4b53-b115-619cd1072043 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:28.197010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb600d-e470-4b53-b115-619cd1072043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:28.197010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb600d-e470-4b53-b115-619cd1072043 admin admin] PATH_INFO: `/v3/projects/3248d63ed2b447aea806395f16188f14/users/0df9bb1a720741768e7bfc1a7b30a8a2/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:28.199089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb600d-e470-4b53-b115-619cd1072043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3248d63ed2b447aea806395f16188f14, user_id=0df9bb1a720741768e7bfc1a7b30a8a2, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:28.206213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb600d-e470-4b53-b115-619cd1072043 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:28.241031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 858/1740] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:28 2022] PUT /identity/v3/projects/3248d63ed2b447aea806395f16188f14/users/0df9bb1a720741768e7bfc1a7b30a8a2/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:28.260167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec33e6b-f7df-4c46-98c8-27a04506270c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:28.260766 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec33e6b-f7df-4c46-98c8-27a04506270c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:28.260858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec33e6b-f7df-4c46-98c8-27a04506270c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:28.370859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2ec33e6b-f7df-4c46-98c8-27a04506270c None None] MFA Rules not processed for user `0df9bb1a720741768e7bfc1a7b30a8a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:28.378955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2ec33e6b-f7df-4c46-98c8-27a04506270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:28.581990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:28.653807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 883/1741] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:28.662537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb600d-e470-4b53-b115-619cd1072043 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:28.666382 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-20cb600d-e470-4b53-b115-619cd1072043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:28.677342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec33e6b-f7df-4c46-98c8-27a04506270c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:28.680585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2ec33e6b-f7df-4c46-98c8-27a04506270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:28.809163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb600d-e470-4b53-b115-619cd1072043 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:28.814769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d090d83-c7f3-4a01-8cff-ec618d0a849f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:28.815940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d090d83-c7f3-4a01-8cff-ec618d0a849f service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:28.816299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d090d83-c7f3-4a01-8cff-ec618d0a849f service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:28.816655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d090d83-c7f3-4a01-8cff-ec618d0a849f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:28.842827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d090d83-c7f3-4a01-8cff-ec618d0a849f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:28.844630 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d090d83-c7f3-4a01-8cff-ec618d0a849f service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:28.860742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 859/1742] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:28.903290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec33e6b-f7df-4c46-98c8-27a04506270c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:28.916412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:28.917373 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:28.917788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:28.918168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] PATH_INFO: `/v3/projects/c8907e8fe5704903835639a7a20b008d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:28.920300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8907e8fe5704903835639a7a20b008d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:28.928908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:28.949805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:29.104672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8907e8fe5704903835639a7a20b008d', 'request_id': 'req-88e49371-ade7-4aaa-bd76-d821832c830a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:29.139016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8907e8fe5704903835639a7a20b008d', 'request_id': 'req-88e49371-ade7-4aaa-bd76-d821832c830a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:29.150730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d090d83-c7f3-4a01-8cff-ec618d0a849f service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:29.152658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2d090d83-c7f3-4a01-8cff-ec618d0a849f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:29.156810 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8907e8fe5704903835639a7a20b008d', 'request_id': 'req-88e49371-ade7-4aaa-bd76-d821832c830a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:29.215680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d090d83-c7f3-4a01-8cff-ec618d0a849f service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:29.251191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] The token cache is being invalidate because project c8907e8fe5704903835639a7a20b008d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:29.255637 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:29.255637 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-88e49371-ade7-4aaa-bd76-d821832c830a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:29.258474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 884/1743] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:28 2022] DELETE /identity/v3/projects/c8907e8fe5704903835639a7a20b008d => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:29.259992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b56898-88cd-429b-89f8-928bb159a063 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:29.261009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b56898-88cd-429b-89f8-928bb159a063 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:29.261598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b56898-88cd-429b-89f8-928bb159a063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:29.300057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c3b56898-88cd-429b-89f8-928bb159a063 None None] MFA Rules not processed for user `12824152139841138f457dfe5c9758e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:29.303026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c3b56898-88cd-429b-89f8-928bb159a063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:29.357255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:29.358244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:29.358720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:29.359052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] PATH_INFO: `/v3/projects/68e7322d628e4ead894ec0c9fb33f55a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:29.366577 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e7322d628e4ead894ec0c9fb33f55a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:29.373922 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:29.522369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 885/1744] 10.222.0.30 () {58 vars in 1015 bytes} [Wed Jul 13 09:06:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:29.527849 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b56898-88cd-429b-89f8-928bb159a063 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:29.530545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c3b56898-88cd-429b-89f8-928bb159a063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:29.579723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b56898-88cd-429b-89f8-928bb159a063 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:29.589963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e7322d628e4ead894ec0c9fb33f55a', 'request_id': 'req-e084744e-5ed3-48f9-9782-8aba585bada9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:29.598764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e7322d628e4ead894ec0c9fb33f55a', 'request_id': 'req-e084744e-5ed3-48f9-9782-8aba585bada9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:29.697090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e7322d628e4ead894ec0c9fb33f55a', 'request_id': 'req-e084744e-5ed3-48f9-9782-8aba585bada9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:29.716282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] The token cache is being invalidate because project 68e7322d628e4ead894ec0c9fb33f55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:29.723039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:29.725912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:29.728922 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 860/1745] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:29 2022] DELETE /identity/v3/projects/68e7322d628e4ead894ec0c9fb33f55a => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:29.730199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:29.736231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:29.768725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:29.768725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:29.768725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:29.777912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] PATH_INFO: `/v3/users/fda6491fd5a144f687f86d9e1f7a7eed` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:29.783287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda6491fd5a144f687f86d9e1f7a7eed)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:29.803463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:29.910999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084744e-5ed3-48f9-9782-8aba585bada9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:29.916573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f899ed-cae5-4972-b425-1d3931ffd594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:29.918666 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f899ed-cae5-4972-b425-1d3931ffd594 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:29.919580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f899ed-cae5-4972-b425-1d3931ffd594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:29.920011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f899ed-cae5-4972-b425-1d3931ffd594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:29.922935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-38f899ed-cae5-4972-b425-1d3931ffd594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:29.942930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f899ed-cae5-4972-b425-1d3931ffd594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:29.942930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f899ed-cae5-4972-b425-1d3931ffd594 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:29.960859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 861/1746] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:29.961449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f899ed-cae5-4972-b425-1d3931ffd594 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:29.964412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-38f899ed-cae5-4972-b425-1d3931ffd594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:29.972538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:29.996270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f899ed-cae5-4972-b425-1d3931ffd594 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:30.004985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b44f3a-0686-4aac-bcbd-728ca77d4c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:30.006932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b44f3a-0686-4aac-bcbd-728ca77d4c9d service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:30.007320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b44f3a-0686-4aac-bcbd-728ca77d4c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:30.022256 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b44f3a-0686-4aac-bcbd-728ca77d4c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:30.048345 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b44f3a-0686-4aac-bcbd-728ca77d4c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:30.052378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b44f3a-0686-4aac-bcbd-728ca77d4c9d service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:30.081758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 862/1747] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:30.083355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41cfda-d007-445b-8de9-6cca468e7d63 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:30.083530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41cfda-d007-445b-8de9-6cca468e7d63 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:30.083783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41cfda-d007-445b-8de9-6cca468e7d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:30.162176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-be41cfda-d007-445b-8de9-6cca468e7d63 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:30.168159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-be41cfda-d007-445b-8de9-6cca468e7d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:30.168663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda6491fd5a144f687f86d9e1f7a7eed'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:30.186583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda6491fd5a144f687f86d9e1f7a7eed'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:30.199980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 863/1748] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:30.202450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dd648-3772-42b7-b1c6-4bce21997629 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:30.202735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dd648-3772-42b7-b1c6-4bce21997629 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:30.208712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722dd648-3772-42b7-b1c6-4bce21997629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:30.240996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:30.262533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-722dd648-3772-42b7-b1c6-4bce21997629 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:30.265506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-722dd648-3772-42b7-b1c6-4bce21997629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:30.306188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 864/1749] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:30.310712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda6491fd5a144f687f86d9e1f7a7eed'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:30.311938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda6491fd5a144f687f86d9e1f7a7eed'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:30.314192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda6491fd5a144f687f86d9e1f7a7eed'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:30.330529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722dd648-3772-42b7-b1c6-4bce21997629 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:30.426656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722dd648-3772-42b7-b1c6-4bce21997629 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:30.445875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 886/1750] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:29 2022] DELETE /identity/v3/users/fda6491fd5a144f687f86d9e1f7a7eed => generated 0 bytes in 923 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:30.458819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a640706-9df2-4589-8c8e-52344cb01f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:30.458819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a640706-9df2-4589-8c8e-52344cb01f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:30.458819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a640706-9df2-4589-8c8e-52344cb01f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:30.472017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a640706-9df2-4589-8c8e-52344cb01f46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:30.472017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a640706-9df2-4589-8c8e-52344cb01f46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:06:30.472017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a640706-9df2-4589-8c8e-52344cb01f46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:30.475262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a640706-9df2-4589-8c8e-52344cb01f46 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:30.505907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 865/1751] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:06:30.510463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:30.512174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:30.541254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:30.691169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa52b6-1fb8-4eb4-bf6c-40add434ca30 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:30.706260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21779e3-77c6-4de4-8a20-9c91a47086fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:30.707106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21779e3-77c6-4de4-8a20-9c91a47086fa admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:30.707488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21779e3-77c6-4de4-8a20-9c91a47086fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:30.707862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21779e3-77c6-4de4-8a20-9c91a47086fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:30.708688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21779e3-77c6-4de4-8a20-9c91a47086fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:30.715631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21779e3-77c6-4de4-8a20-9c91a47086fa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:30.857569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 887/1752] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:30 2022] POST /identity/v3/projects => generated 373 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:30.869569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a640706-9df2-4589-8c8e-52344cb01f46 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:30.881436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21779e3-77c6-4de4-8a20-9c91a47086fa admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:30.883716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e21779e3-77c6-4de4-8a20-9c91a47086fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:30.889468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a640706-9df2-4589-8c8e-52344cb01f46 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:31.005805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d19c8-32f1-4b36-9459-2009580628e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:31.005805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d19c8-32f1-4b36-9459-2009580628e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:31.005805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d19c8-32f1-4b36-9459-2009580628e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:31.005805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d19c8-32f1-4b36-9459-2009580628e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:31.006311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d19c8-32f1-4b36-9459-2009580628e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:31.006311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d19c8-32f1-4b36-9459-2009580628e9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:31.043204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21779e3-77c6-4de4-8a20-9c91a47086fa admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:31.052620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e6915-af7f-48a3-94f7-c6a7432c796a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:31.057002 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e6915-af7f-48a3-94f7-c6a7432c796a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:31.057296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e6915-af7f-48a3-94f7-c6a7432c796a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:31.059529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e6915-af7f-48a3-94f7-c6a7432c796a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:31.070152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f31e6915-af7f-48a3-94f7-c6a7432c796a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:31.178085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e6915-af7f-48a3-94f7-c6a7432c796a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:31.180188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e6915-af7f-48a3-94f7-c6a7432c796a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:31.222050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 888/1753] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:06:31.311555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 866/1754] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:30 2022] POST /identity/v3/users => generated 287 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:06:31.316679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e6915-af7f-48a3-94f7-c6a7432c796a service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:31.332591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e6915-af7f-48a3-94f7-c6a7432c796a service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:31.368084 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c6d2-b429-46b6-8c57-e5a8bd6c3ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:31.368781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c6d2-b429-46b6-8c57-e5a8bd6c3ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:31.369078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c6d2-b429-46b6-8c57-e5a8bd6c3ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:31.369346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c6d2-b429-46b6-8c57-e5a8bd6c3ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:31.370662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4c6d2-b429-46b6-8c57-e5a8bd6c3ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:31.373882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4c6d2-b429-46b6-8c57-e5a8bd6c3ee4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:31.403101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d19c8-32f1-4b36-9459-2009580628e9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:31.410369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 889/1755] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:06:31.418017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d19c8-32f1-4b36-9459-2009580628e9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:31.431462 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f297ec-1279-4d6c-87cd-4f7aab5b1fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:31.432623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f297ec-1279-4d6c-87cd-4f7aab5b1fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:31.432911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f297ec-1279-4d6c-87cd-4f7aab5b1fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:31.433189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f297ec-1279-4d6c-87cd-4f7aab5b1fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:31.433943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f297ec-1279-4d6c-87cd-4f7aab5b1fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:31.435762 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f297ec-1279-4d6c-87cd-4f7aab5b1fb1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:31.473500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 867/1756] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:31.473708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c6d2-b429-46b6-8c57-e5a8bd6c3ee4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:31.488179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c6d2-b429-46b6-8c57-e5a8bd6c3ee4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:31.519152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788977f-50bf-4999-92c5-a4770dfc4a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:31.520099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788977f-50bf-4999-92c5-a4770dfc4a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:31.520456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788977f-50bf-4999-92c5-a4770dfc4a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:31.520747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788977f-50bf-4999-92c5-a4770dfc4a54 admin admin] PATH_INFO: `/v3/projects/dae36c8704fc4fa8a69d9b58077460e4/users/b03791c8d0ad4811a5cf28ea916132e7/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:31.583029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788977f-50bf-4999-92c5-a4770dfc4a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae36c8704fc4fa8a69d9b58077460e4, user_id=b03791c8d0ad4811a5cf28ea916132e7, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:31.588950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788977f-50bf-4999-92c5-a4770dfc4a54 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:31.682578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 890/1757] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:31 2022] PUT /identity/v3/projects/dae36c8704fc4fa8a69d9b58077460e4/users/b03791c8d0ad4811a5cf28ea916132e7/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:31.697052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4db8-6a16-41c7-9ed3-4ad3ac17abb1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:31.697718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4db8-6a16-41c7-9ed3-4ad3ac17abb1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:31.698009 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4db8-6a16-41c7-9ed3-4ad3ac17abb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:31.802978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3c1c4db8-6a16-41c7-9ed3-4ad3ac17abb1 None None] MFA Rules not processed for user `b03791c8d0ad4811a5cf28ea916132e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:31.802978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3c1c4db8-6a16-41c7-9ed3-4ad3ac17abb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:32.105687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 868/1758] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:32.110156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788977f-50bf-4999-92c5-a4770dfc4a54 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:32.270953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788977f-50bf-4999-92c5-a4770dfc4a54 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:32.289894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb44f20-f8b9-4068-9733-731f8b89fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:32.290315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb44f20-f8b9-4068-9733-731f8b89fa62 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:32.290598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb44f20-f8b9-4068-9733-731f8b89fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:32.290875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb44f20-f8b9-4068-9733-731f8b89fa62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:32.291575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb44f20-f8b9-4068-9733-731f8b89fa62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:32.293488 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb44f20-f8b9-4068-9733-731f8b89fa62 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:32.492077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 891/1759] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:32 2022] POST /identity/v3/projects => generated 371 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:32.507152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c4db8-6a16-41c7-9ed3-4ad3ac17abb1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:32.532791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c4db8-6a16-41c7-9ed3-4ad3ac17abb1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:32.726697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668525a-ba3f-4b20-9bf1-5f413a58061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:32.730525 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668525a-ba3f-4b20-9bf1-5f413a58061a admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:32.730800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668525a-ba3f-4b20-9bf1-5f413a58061a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:32.742853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668525a-ba3f-4b20-9bf1-5f413a58061a admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:32.742853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668525a-ba3f-4b20-9bf1-5f413a58061a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:32.742853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668525a-ba3f-4b20-9bf1-5f413a58061a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:32.868001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:32.920338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb44f20-f8b9-4068-9733-731f8b89fa62 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:32.922685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 869/1760] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:32 2022] POST /identity/v3/users => generated 292 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:32.941231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb44f20-f8b9-4068-9733-731f8b89fa62 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:32.964418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f735e94-a359-4ce8-9cf9-27cf46177153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:32.968320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f735e94-a359-4ce8-9cf9-27cf46177153 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:32.968695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f735e94-a359-4ce8-9cf9-27cf46177153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:32.969072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f735e94-a359-4ce8-9cf9-27cf46177153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:32.970166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f735e94-a359-4ce8-9cf9-27cf46177153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:32.972544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f735e94-a359-4ce8-9cf9-27cf46177153 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:33.008158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 892/1761] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:06:33.010980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668525a-ba3f-4b20-9bf1-5f413a58061a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:33.032909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668525a-ba3f-4b20-9bf1-5f413a58061a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:33.032909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b8337-727b-43b8-a42e-18cec969a68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:33.032909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b8337-727b-43b8-a42e-18cec969a68a admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:33.032909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b8337-727b-43b8-a42e-18cec969a68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:33.033860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b8337-727b-43b8-a42e-18cec969a68a admin admin] PATH_INFO: `/v3/projects/ff8d353b65864c36a462fc8a24c4ee6c/users/d79e4ef6a4794ebca02353a47c91676b/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:33.075627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b8337-727b-43b8-a42e-18cec969a68a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8d353b65864c36a462fc8a24c4ee6c, user_id=d79e4ef6a4794ebca02353a47c91676b, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:33.079064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b8337-727b-43b8-a42e-18cec969a68a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:33.158122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f735e94-a359-4ce8-9cf9-27cf46177153 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:33.162332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5f735e94-a359-4ce8-9cf9-27cf46177153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:33.175658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:33.178964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 870/1762] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:33 2022] PUT /identity/v3/projects/ff8d353b65864c36a462fc8a24c4ee6c/users/d79e4ef6a4794ebca02353a47c91676b/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:33.186213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b8337-727b-43b8-a42e-18cec969a68a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:33.202993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f735e94-a359-4ce8-9cf9-27cf46177153 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:33.470433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:33.476394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:33.476715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:33.481145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] PATH_INFO: `/v3/projects/03dfa33ae39d4d5e8c2d7d2db7735070` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:33.484250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=03dfa33ae39d4d5e8c2d7d2db7735070)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:33.486607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:33.523227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b8337-727b-43b8-a42e-18cec969a68a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:33.553660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef76dd2-6149-4f78-979a-8daca27707d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:33.554701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef76dd2-6149-4f78-979a-8daca27707d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:33.555131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef76dd2-6149-4f78-979a-8daca27707d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:33.555842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef76dd2-6149-4f78-979a-8daca27707d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:33.556795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef76dd2-6149-4f78-979a-8daca27707d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:33.559077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef76dd2-6149-4f78-979a-8daca27707d3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:33.586411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 871/1763] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:33.590573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef76dd2-6149-4f78-979a-8daca27707d3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:33.633400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef76dd2-6149-4f78-979a-8daca27707d3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:33.662086 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11de72-da51-42c4-ac5f-dcddf2bbb06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:33.666309 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11de72-da51-42c4-ac5f-dcddf2bbb06f admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:33.666309 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11de72-da51-42c4-ac5f-dcddf2bbb06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:33.666309 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11de72-da51-42c4-ac5f-dcddf2bbb06f admin admin] PATH_INFO: `/v3/projects/ff8d353b65864c36a462fc8a24c4ee6c/users/d79e4ef6a4794ebca02353a47c91676b/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:33.675397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11de72-da51-42c4-ac5f-dcddf2bbb06f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8d353b65864c36a462fc8a24c4ee6c, user_id=d79e4ef6a4794ebca02353a47c91676b, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:33.681972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11de72-da51-42c4-ac5f-dcddf2bbb06f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:33.757679 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 872/1764] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:33 2022] PUT /identity/v3/projects/ff8d353b65864c36a462fc8a24c4ee6c/users/d79e4ef6a4794ebca02353a47c91676b/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:33.792145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff88fe6-024d-4acb-a1ba-d19ff6aefac8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:33.794219 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff88fe6-024d-4acb-a1ba-d19ff6aefac8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:33.794219 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff88fe6-024d-4acb-a1ba-d19ff6aefac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:33.846600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03dfa33ae39d4d5e8c2d7d2db7735070', 'request_id': 'req-7503d866-625c-472f-b793-e34d7ef35b4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:33.857164 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03dfa33ae39d4d5e8c2d7d2db7735070', 'request_id': 'req-7503d866-625c-472f-b793-e34d7ef35b4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:33.877767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03dfa33ae39d4d5e8c2d7d2db7735070', 'request_id': 'req-7503d866-625c-472f-b793-e34d7ef35b4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:33.913476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-dff88fe6-024d-4acb-a1ba-d19ff6aefac8 None None] MFA Rules not processed for user `d79e4ef6a4794ebca02353a47c91676b`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:33.915059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dff88fe6-024d-4acb-a1ba-d19ff6aefac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:33.997042 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] The token cache is being invalidate because project 03dfa33ae39d4d5e8c2d7d2db7735070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:33.997711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:34.000558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7503d866-625c-472f-b793-e34d7ef35b4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:34.014503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 893/1765] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:33 2022] DELETE /identity/v3/projects/03dfa33ae39d4d5e8c2d7d2db7735070 => generated 0 bytes in 857 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:34.027492 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59767de1-9e2f-4b8a-bcbb-bf259e07e75b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:34.027786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59767de1-9e2f-4b8a-bcbb-bf259e07e75b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:34.028693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59767de1-9e2f-4b8a-bcbb-bf259e07e75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:34.082811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-59767de1-9e2f-4b8a-bcbb-bf259e07e75b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:34.088355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-59767de1-9e2f-4b8a-bcbb-bf259e07e75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:34.253413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 873/1766] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:33 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:34.295147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 894/1767] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:34.300445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486f074-1040-4697-9ccd-9b3d2f899965 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:34.300815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486f074-1040-4697-9ccd-9b3d2f899965 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:34.301097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486f074-1040-4697-9ccd-9b3d2f899965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:34.326485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59767de1-9e2f-4b8a-bcbb-bf259e07e75b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:34.328707 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-59767de1-9e2f-4b8a-bcbb-bf259e07e75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:34.365132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6486f074-1040-4697-9ccd-9b3d2f899965 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:34.369132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6486f074-1040-4697-9ccd-9b3d2f899965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:34.407702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 874/1768] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:34.409514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486f074-1040-4697-9ccd-9b3d2f899965 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:34.433851 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486f074-1040-4697-9ccd-9b3d2f899965 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:34.459547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced5a7e-2794-4fd3-8ea2-c4dee6becac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:34.460306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced5a7e-2794-4fd3-8ea2-c4dee6becac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:34.460586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced5a7e-2794-4fd3-8ea2-c4dee6becac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:34.460864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced5a7e-2794-4fd3-8ea2-c4dee6becac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:34.463567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced5a7e-2794-4fd3-8ea2-c4dee6becac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:06:34.463860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced5a7e-2794-4fd3-8ea2-c4dee6becac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:34.465595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced5a7e-2794-4fd3-8ea2-c4dee6becac8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:34.510090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 875/1769] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:06:34.525398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59767de1-9e2f-4b8a-bcbb-bf259e07e75b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:34.528094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced5a7e-2794-4fd3-8ea2-c4dee6becac8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:34.545072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced5a7e-2794-4fd3-8ea2-c4dee6becac8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:34.568910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12549b2e-51e0-403e-85d1-7f18d1488230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:34.570004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12549b2e-51e0-403e-85d1-7f18d1488230 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:34.570920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12549b2e-51e0-403e-85d1-7f18d1488230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:34.572319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12549b2e-51e0-403e-85d1-7f18d1488230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:34.572319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12549b2e-51e0-403e-85d1-7f18d1488230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:34.575258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6696488-b79b-408c-9c09-efcbb7960c2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:34.576847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6696488-b79b-408c-9c09-efcbb7960c2a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:34.577998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6696488-b79b-408c-9c09-efcbb7960c2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:34.579208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6696488-b79b-408c-9c09-efcbb7960c2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:34.579332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12549b2e-51e0-403e-85d1-7f18d1488230 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:34.600617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6696488-b79b-408c-9c09-efcbb7960c2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:34.604545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6696488-b79b-408c-9c09-efcbb7960c2a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:34.634169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 895/1770] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:34.646860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:34.657314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cdd93-14ea-4db6-a30e-7154b2f9960e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:34.659435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cdd93-14ea-4db6-a30e-7154b2f9960e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:34.660048 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cdd93-14ea-4db6-a30e-7154b2f9960e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:34.666199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 896/1771] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 09:06:34 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:06:34.704468 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df33a-e18d-42ec-8511-f069d7a2e36b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:34.708215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df33a-e18d-42ec-8511-f069d7a2e36b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:34.715291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df33a-e18d-42ec-8511-f069d7a2e36b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:34.723976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 897/1772] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 09:06:34 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:06:34.739126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7602e4-c59b-4bc0-b66f-975a9c9abd08 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:34.739126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7602e4-c59b-4bc0-b66f-975a9c9abd08 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:34.739126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7602e4-c59b-4bc0-b66f-975a9c9abd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:34.745593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 876/1773] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:34 2022] POST /identity/v3/projects => generated 363 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:34.748367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12549b2e-51e0-403e-85d1-7f18d1488230 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:34.811142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bb7602e4-c59b-4bc0-b66f-975a9c9abd08 None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:34.812251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bb7602e4-c59b-4bc0-b66f-975a9c9abd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:35.005690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12549b2e-51e0-403e-85d1-7f18d1488230 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:35.015635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f8d3e-90da-4de2-a479-2ee79c2cd287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:35.016443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f8d3e-90da-4de2-a479-2ee79c2cd287 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:35.016788 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f8d3e-90da-4de2-a479-2ee79c2cd287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:35.017411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f8d3e-90da-4de2-a479-2ee79c2cd287 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:35.018284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f8d3e-90da-4de2-a479-2ee79c2cd287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:35.020694 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f8d3e-90da-4de2-a479-2ee79c2cd287 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:35.176381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 898/1774] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 09:06:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:35.190767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316259ef-a4eb-414d-8b61-59a811869543 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:35.191086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316259ef-a4eb-414d-8b61-59a811869543 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:35.191400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316259ef-a4eb-414d-8b61-59a811869543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:35.227188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-316259ef-a4eb-414d-8b61-59a811869543 None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:35.229219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-316259ef-a4eb-414d-8b61-59a811869543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:35.244479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 899/1775] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 09:06:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:35.255674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 877/1776] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:34 2022] POST /identity/v3/users => generated 282 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:35.259906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316259ef-a4eb-414d-8b61-59a811869543 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:35.272599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316259ef-a4eb-414d-8b61-59a811869543 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:35.304670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06343d-eb54-4b5b-ab27-e9db7d1eec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:35.304670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06343d-eb54-4b5b-ab27-e9db7d1eec7c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:35.304670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06343d-eb54-4b5b-ab27-e9db7d1eec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:35.304670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06343d-eb54-4b5b-ab27-e9db7d1eec7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:35.305124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06343d-eb54-4b5b-ab27-e9db7d1eec7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:35.305124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06343d-eb54-4b5b-ab27-e9db7d1eec7c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:35.305124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 900/1777] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:35.306162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f8d3e-90da-4de2-a479-2ee79c2cd287 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:35.323471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f8d3e-90da-4de2-a479-2ee79c2cd287 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:35.331386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095254-4e7b-456a-882b-e621d4655d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:35.332507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095254-4e7b-456a-882b-e621d4655d0b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:35.332806 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095254-4e7b-456a-882b-e621d4655d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:35.333101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095254-4e7b-456a-882b-e621d4655d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:35.334154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57095254-4e7b-456a-882b-e621d4655d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:35.361386 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57095254-4e7b-456a-882b-e621d4655d0b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:35.412888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 878/1778] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:35.420268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06343d-eb54-4b5b-ab27-e9db7d1eec7c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:35.436187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06343d-eb54-4b5b-ab27-e9db7d1eec7c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:35.452587 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95706f5-7d7d-40d2-8593-ec1588f60013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:35.453569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95706f5-7d7d-40d2-8593-ec1588f60013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:35.453978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95706f5-7d7d-40d2-8593-ec1588f60013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:35.454299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95706f5-7d7d-40d2-8593-ec1588f60013 admin admin] PATH_INFO: `/v3/projects/04254b9b7bdc426382c0232f5aedd038/users/542c321896f94802b0acf9be1e6d138b/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:35.496177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95706f5-7d7d-40d2-8593-ec1588f60013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04254b9b7bdc426382c0232f5aedd038, user_id=542c321896f94802b0acf9be1e6d138b, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:35.499796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95706f5-7d7d-40d2-8593-ec1588f60013 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:35.813740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095254-4e7b-456a-882b-e621d4655d0b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:35.816317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-57095254-4e7b-456a-882b-e621d4655d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:35.862264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095254-4e7b-456a-882b-e621d4655d0b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:35.867438 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2559d-d11c-4d1e-ae99-3da1b8fdaef4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:35.868529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2559d-d11c-4d1e-ae99-3da1b8fdaef4 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:35.869022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2559d-d11c-4d1e-ae99-3da1b8fdaef4 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:35.869491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2559d-d11c-4d1e-ae99-3da1b8fdaef4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:35.886395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2559d-d11c-4d1e-ae99-3da1b8fdaef4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:35.888191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2559d-d11c-4d1e-ae99-3da1b8fdaef4 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:35.914461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 901/1779] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:35 2022] PUT /identity/v3/projects/04254b9b7bdc426382c0232f5aedd038/users/542c321896f94802b0acf9be1e6d138b/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:35.925474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 879/1780] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:35.927187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb5f20-c566-4ce2-975e-0a2ff6e90bd9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:35.927477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb5f20-c566-4ce2-975e-0a2ff6e90bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:35.927763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb5f20-c566-4ce2-975e-0a2ff6e90bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:35.964534 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2559d-d11c-4d1e-ae99-3da1b8fdaef4 service nova] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:35.966344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-93e2559d-d11c-4d1e-ae99-3da1b8fdaef4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:36.006796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2559d-d11c-4d1e-ae99-3da1b8fdaef4 service nova] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:36.029530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-eefb5f20-c566-4ce2-975e-0a2ff6e90bd9 None None] MFA Rules not processed for user `542c321896f94802b0acf9be1e6d138b`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:36.039651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-eefb5f20-c566-4ce2-975e-0a2ff6e90bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:36.171370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886f275-6a8f-4c66-b920-0087b34103d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:36.172616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886f275-6a8f-4c66-b920-0087b34103d6 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:36.173368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886f275-6a8f-4c66-b920-0087b34103d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:36.174860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886f275-6a8f-4c66-b920-0087b34103d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:36.199614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886f275-6a8f-4c66-b920-0087b34103d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:36.202052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886f275-6a8f-4c66-b920-0087b34103d6 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:36.211498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:36.214978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 880/1781] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:06:36.265394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 902/1782] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:36.265502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886f275-6a8f-4c66-b920-0087b34103d6 service nova] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:36.479441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886f275-6a8f-4c66-b920-0087b34103d6 service nova] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:36.488855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30d18e-53e6-4583-8f2e-2b7dea984a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:36.489494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d18e-53e6-4583-8f2e-2b7dea984a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:36.489912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d18e-53e6-4583-8f2e-2b7dea984a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:36.490204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d18e-53e6-4583-8f2e-2b7dea984a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:36.491217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30d18e-53e6-4583-8f2e-2b7dea984a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:36.493807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30d18e-53e6-4583-8f2e-2b7dea984a48 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:36.633990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 881/1783] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:36 2022] POST /identity/v3/projects => generated 363 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:36.635617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb5f20-c566-4ce2-975e-0a2ff6e90bd9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:36.657414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb5f20-c566-4ce2-975e-0a2ff6e90bd9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:36.810098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1b2f2-fff2-4cb1-abc9-6d0c25cd9b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:36.811317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1b2f2-fff2-4cb1-abc9-6d0c25cd9b03 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:36.811750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1b2f2-fff2-4cb1-abc9-6d0c25cd9b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:36.812257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1b2f2-fff2-4cb1-abc9-6d0c25cd9b03 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:36.813316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1b2f2-fff2-4cb1-abc9-6d0c25cd9b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:36.816437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1b2f2-fff2-4cb1-abc9-6d0c25cd9b03 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:36.953318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 903/1784] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:36 2022] POST /identity/v3/users => generated 288 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:36.953519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30d18e-53e6-4583-8f2e-2b7dea984a48 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:36.963262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30d18e-53e6-4583-8f2e-2b7dea984a48 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:36.979216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa22e-4bc6-4c27-acbe-970324bb4f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:36.979945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa22e-4bc6-4c27-acbe-970324bb4f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:36.980234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa22e-4bc6-4c27-acbe-970324bb4f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:36.980540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa22e-4bc6-4c27-acbe-970324bb4f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:36.981226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa22e-4bc6-4c27-acbe-970324bb4f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:36.983039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa22e-4bc6-4c27-acbe-970324bb4f01 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:37.015205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 882/1785] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:06:37.020251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1b2f2-fff2-4cb1-abc9-6d0c25cd9b03 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:37.047149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1b2f2-fff2-4cb1-abc9-6d0c25cd9b03 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:37.057610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be75b6-aa5e-4ae1-93c6-e46c81fb1d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:37.065616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be75b6-aa5e-4ae1-93c6-e46c81fb1d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:37.065616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be75b6-aa5e-4ae1-93c6-e46c81fb1d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:37.065616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be75b6-aa5e-4ae1-93c6-e46c81fb1d6f admin admin] PATH_INFO: `/v3/projects/c9318989c1864a25b8aea5c99d179931/users/e65383f5aa0d4dc39c7eb14a51a4e1ca/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:37.138946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be75b6-aa5e-4ae1-93c6-e46c81fb1d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9318989c1864a25b8aea5c99d179931, user_id=e65383f5aa0d4dc39c7eb14a51a4e1ca, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:37.147401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be75b6-aa5e-4ae1-93c6-e46c81fb1d6f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:37.253399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 904/1786] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:37 2022] PUT /identity/v3/projects/c9318989c1864a25b8aea5c99d179931/users/e65383f5aa0d4dc39c7eb14a51a4e1ca/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:37.257085 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa22e-4bc6-4c27-acbe-970324bb4f01 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:37.302958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa22e-4bc6-4c27-acbe-970324bb4f01 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:37.369257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:37.447345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be75b6-aa5e-4ae1-93c6-e46c81fb1d6f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:37.449515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-46be75b6-aa5e-4ae1-93c6-e46c81fb1d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:37.466333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803f730-a1fa-46de-a1f6-fce8bda0107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:37.467755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803f730-a1fa-46de-a1f6-fce8bda0107b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:37.467755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803f730-a1fa-46de-a1f6-fce8bda0107b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:37.474284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803f730-a1fa-46de-a1f6-fce8bda0107b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:37.475160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803f730-a1fa-46de-a1f6-fce8bda0107b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:37.480257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803f730-a1fa-46de-a1f6-fce8bda0107b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:37.516604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 883/1787] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:37.539756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803f730-a1fa-46de-a1f6-fce8bda0107b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:37.553517 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:37.557748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803f730-a1fa-46de-a1f6-fce8bda0107b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:37.567318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be75b6-aa5e-4ae1-93c6-e46c81fb1d6f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:37.573895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:37.577109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:37.577109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:37.577109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] PATH_INFO: `/v3/users/b03791c8d0ad4811a5cf28ea916132e7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:37.578450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b03791c8d0ad4811a5cf28ea916132e7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:37.581830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:37.583537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f0ed0-fb72-448d-a3d2-4e5865f6a992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:37.584489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f0ed0-fb72-448d-a3d2-4e5865f6a992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:37.584784 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f0ed0-fb72-448d-a3d2-4e5865f6a992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:37.585451 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f0ed0-fb72-448d-a3d2-4e5865f6a992 admin admin] PATH_INFO: `/v3/projects/c9318989c1864a25b8aea5c99d179931/users/e65383f5aa0d4dc39c7eb14a51a4e1ca/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:37.591740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f0ed0-fb72-448d-a3d2-4e5865f6a992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9318989c1864a25b8aea5c99d179931, user_id=e65383f5aa0d4dc39c7eb14a51a4e1ca, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:37.596912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f0ed0-fb72-448d-a3d2-4e5865f6a992 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:37.632186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 884/1788] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:37 2022] PUT /identity/v3/projects/c9318989c1864a25b8aea5c99d179931/users/e65383f5aa0d4dc39c7eb14a51a4e1ca/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:37.644360 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4788e-9f1b-468a-bd7b-d65077b5244c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:37.644676 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4788e-9f1b-468a-bd7b-d65077b5244c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:37.644902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4788e-9f1b-468a-bd7b-d65077b5244c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:37.714775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7cf4788e-9f1b-468a-bd7b-d65077b5244c None None] MFA Rules not processed for user `e65383f5aa0d4dc39c7eb14a51a4e1ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:37.717418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7cf4788e-9f1b-468a-bd7b-d65077b5244c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:37.802066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b03791c8d0ad4811a5cf28ea916132e7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:37.837974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b03791c8d0ad4811a5cf28ea916132e7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:37.865585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 885/1789] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:37 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:37.883624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4788e-9f1b-468a-bd7b-d65077b5244c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:37.920778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b03791c8d0ad4811a5cf28ea916132e7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:37.922085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b03791c8d0ad4811a5cf28ea916132e7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:37.923477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f9115f3-e5b2-449b-96a1-853607361884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b03791c8d0ad4811a5cf28ea916132e7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:37.962921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 905/1790] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:37 2022] DELETE /identity/v3/users/b03791c8d0ad4811a5cf28ea916132e7 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:37.969131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff892d8-68e0-4136-8df0-81e3a840c046 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:37.969131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff892d8-68e0-4136-8df0-81e3a840c046 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:37.969131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff892d8-68e0-4136-8df0-81e3a840c046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:38.055783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4788e-9f1b-468a-bd7b-d65077b5244c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:38.071619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-cff892d8-68e0-4136-8df0-81e3a840c046 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:38.077992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cff892d8-68e0-4136-8df0-81e3a840c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:38.186935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac4b7d-789a-4dd0-b886-eada660ccaeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:38.187829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac4b7d-789a-4dd0-b886-eada660ccaeb service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:38.188279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac4b7d-789a-4dd0-b886-eada660ccaeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:38.188725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac4b7d-789a-4dd0-b886-eada660ccaeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:38.190911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b0ac4b7d-789a-4dd0-b886-eada660ccaeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:38.266406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac4b7d-789a-4dd0-b886-eada660ccaeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:38.268941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac4b7d-789a-4dd0-b886-eada660ccaeb service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:38.301864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 886/1791] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:37 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:38.301864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac4b7d-789a-4dd0-b886-eada660ccaeb service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:38.301864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b0ac4b7d-789a-4dd0-b886-eada660ccaeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:38.313757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac4b7d-789a-4dd0-b886-eada660ccaeb service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:38.322678 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 906/1792] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:38.329536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e8369-77a5-4e78-80f9-c5e76175a56a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:38.330818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e8369-77a5-4e78-80f9-c5e76175a56a service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:38.331100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e8369-77a5-4e78-80f9-c5e76175a56a service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:38.331467 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e8369-77a5-4e78-80f9-c5e76175a56a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:38.333482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-310e8369-77a5-4e78-80f9-c5e76175a56a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:38.416826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff892d8-68e0-4136-8df0-81e3a840c046 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:38.420088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cff892d8-68e0-4136-8df0-81e3a840c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:38.445775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e8369-77a5-4e78-80f9-c5e76175a56a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:38.447798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e8369-77a5-4e78-80f9-c5e76175a56a service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:38.475212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 887/1793] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:38.571441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff892d8-68e0-4136-8df0-81e3a840c046 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:38.581925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d0628-5ded-46c3-8f31-96018febb87e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:38.582665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d0628-5ded-46c3-8f31-96018febb87e service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:38.582951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d0628-5ded-46c3-8f31-96018febb87e service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:38.583432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d0628-5ded-46c3-8f31-96018febb87e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:38.599968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d0628-5ded-46c3-8f31-96018febb87e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:38.601780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d0628-5ded-46c3-8f31-96018febb87e service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:38.609649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e8369-77a5-4e78-80f9-c5e76175a56a service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:38.611605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-310e8369-77a5-4e78-80f9-c5e76175a56a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:38.618063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 907/1794] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:38.643777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e8369-77a5-4e78-80f9-c5e76175a56a service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:38.664037 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b325d8d-cf62-4abd-a1bc-093f2328c0cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:38.666701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b325d8d-cf62-4abd-a1bc-093f2328c0cd service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:38.669904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b325d8d-cf62-4abd-a1bc-093f2328c0cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:38.669904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b325d8d-cf62-4abd-a1bc-093f2328c0cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:38.676646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0b325d8d-cf62-4abd-a1bc-093f2328c0cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:38.803696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b325d8d-cf62-4abd-a1bc-093f2328c0cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:38.806025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b325d8d-cf62-4abd-a1bc-093f2328c0cd service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:38.821337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 888/1795] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:38 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:39.228357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:39.514136 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:40.585897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d0628-5ded-46c3-8f31-96018febb87e service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:40.588112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-519d0628-5ded-46c3-8f31-96018febb87e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:40.609570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d0628-5ded-46c3-8f31-96018febb87e service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:40.620226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:40.621471 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:40.621951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:40.622318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] PATH_INFO: `/v3/projects/dae36c8704fc4fa8a69d9b58077460e4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:40.626230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae36c8704fc4fa8a69d9b58077460e4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:40.636659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:40.900893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae36c8704fc4fa8a69d9b58077460e4', 'request_id': 'req-857b7572-e4f6-467d-858e-ba905a76c7c0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:40.907782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae36c8704fc4fa8a69d9b58077460e4', 'request_id': 'req-857b7572-e4f6-467d-858e-ba905a76c7c0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:40.916981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae36c8704fc4fa8a69d9b58077460e4', 'request_id': 'req-857b7572-e4f6-467d-858e-ba905a76c7c0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:40.999967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] The token cache is being invalidate because project dae36c8704fc4fa8a69d9b58077460e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:41.001007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:41.002596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:41.007393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 908/1796] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:40 2022] DELETE /identity/v3/projects/dae36c8704fc4fa8a69d9b58077460e4 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:41.009717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b325d8d-cf62-4abd-a1bc-093f2328c0cd service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:41.053084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b325d8d-cf62-4abd-a1bc-093f2328c0cd service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:41.193013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:41.194171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:41.194742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:41.195111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] PATH_INFO: `/v3/users/d79e4ef6a4794ebca02353a47c91676b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:41.196943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79e4ef6a4794ebca02353a47c91676b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:41.199554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:41.297206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:41.676153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79e4ef6a4794ebca02353a47c91676b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:41.686755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79e4ef6a4794ebca02353a47c91676b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:41.703233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79e4ef6a4794ebca02353a47c91676b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:41.708123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:41.826375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79e4ef6a4794ebca02353a47c91676b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:41.830415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e32c981e-6233-4645-bb71-997c4a2d357a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79e4ef6a4794ebca02353a47c91676b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:41.835393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 889/1797] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:41 2022] DELETE /identity/v3/users/d79e4ef6a4794ebca02353a47c91676b => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:42.413489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:42.413489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:42.428858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b7572-e4f6-467d-858e-ba905a76c7c0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:42.539162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:42.540164 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:42.540524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:42.540911 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] PATH_INFO: `/v3/projects/ff8d353b65864c36a462fc8a24c4ee6c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:42.542386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8d353b65864c36a462fc8a24c4ee6c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:42.544515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:42.667915 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8d353b65864c36a462fc8a24c4ee6c', 'request_id': 'req-300a3e9b-21e6-4c16-9f3e-4819f7645317'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:42.674977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8d353b65864c36a462fc8a24c4ee6c', 'request_id': 'req-300a3e9b-21e6-4c16-9f3e-4819f7645317'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:42.681423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8d353b65864c36a462fc8a24c4ee6c', 'request_id': 'req-300a3e9b-21e6-4c16-9f3e-4819f7645317'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:42.787625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] The token cache is being invalidate because project ff8d353b65864c36a462fc8a24c4ee6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:42.788313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:42.788690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-300a3e9b-21e6-4c16-9f3e-4819f7645317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:42.790462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 909/1798] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:42 2022] DELETE /identity/v3/projects/ff8d353b65864c36a462fc8a24c4ee6c => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:42.795476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15052a-56b6-4953-84c8-b083a6dc9fbf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:42.795954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15052a-56b6-4953-84c8-b083a6dc9fbf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:42.796149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15052a-56b6-4953-84c8-b083a6dc9fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:42.815336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bd15052a-56b6-4953-84c8-b083a6dc9fbf None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:42.816486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bd15052a-56b6-4953-84c8-b083a6dc9fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:42.874056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 890/1799] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:42.883607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b210d4-ea0b-41b4-bb20-d568111791e6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:42.883892 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b210d4-ea0b-41b4-bb20-d568111791e6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:42.884168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b210d4-ea0b-41b4-bb20-d568111791e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:42.902455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-70b210d4-ea0b-41b4-bb20-d568111791e6 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:42.903635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-70b210d4-ea0b-41b4-bb20-d568111791e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:42.911402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 910/1800] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:42.915726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15052a-56b6-4953-84c8-b083a6dc9fbf None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:42.925842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15052a-56b6-4953-84c8-b083a6dc9fbf None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:42.930616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3be1c-2ddd-46d9-bacc-a4829e350ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:42.931271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3be1c-2ddd-46d9-bacc-a4829e350ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:42.931550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3be1c-2ddd-46d9-bacc-a4829e350ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:42.931821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3be1c-2ddd-46d9-bacc-a4829e350ad3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:42.932536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3be1c-2ddd-46d9-bacc-a4829e350ad3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:06:42.932842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3be1c-2ddd-46d9-bacc-a4829e350ad3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:42.934561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3be1c-2ddd-46d9-bacc-a4829e350ad3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:42.946506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 891/1801] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:06:42.950829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b210d4-ea0b-41b4-bb20-d568111791e6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:42.960071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b210d4-ea0b-41b4-bb20-d568111791e6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:42.964730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726609b5-96b4-4c5d-b507-8caae82d23e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:42.965398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726609b5-96b4-4c5d-b507-8caae82d23e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:42.965765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726609b5-96b4-4c5d-b507-8caae82d23e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:42.966062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726609b5-96b4-4c5d-b507-8caae82d23e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:42.966762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726609b5-96b4-4c5d-b507-8caae82d23e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:42.968652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726609b5-96b4-4c5d-b507-8caae82d23e6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:43.154869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 911/1802] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:42 2022] POST /identity/v3/projects => generated 377 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:43.159279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3be1c-2ddd-46d9-bacc-a4829e350ad3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:43.171701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3be1c-2ddd-46d9-bacc-a4829e350ad3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:43.241370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c906cc8-094e-49be-b80b-77d196bd50a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:43.242149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c906cc8-094e-49be-b80b-77d196bd50a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:43.243163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c906cc8-094e-49be-b80b-77d196bd50a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:43.243579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c906cc8-094e-49be-b80b-77d196bd50a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:43.244282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c906cc8-094e-49be-b80b-77d196bd50a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:43.246227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c906cc8-094e-49be-b80b-77d196bd50a9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:43.384647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 892/1803] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:43 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:43.388400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726609b5-96b4-4c5d-b507-8caae82d23e6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:43.401598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726609b5-96b4-4c5d-b507-8caae82d23e6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:43.408750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef78cb-c23a-4eb8-b18a-4759b597a1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:43.409434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef78cb-c23a-4eb8-b18a-4759b597a1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:43.409716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef78cb-c23a-4eb8-b18a-4759b597a1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:43.409993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef78cb-c23a-4eb8-b18a-4759b597a1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:43.410749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef78cb-c23a-4eb8-b18a-4759b597a1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:43.412352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef78cb-c23a-4eb8-b18a-4759b597a1d2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:43.425138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 912/1804] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:43.427654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c906cc8-094e-49be-b80b-77d196bd50a9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:43.438048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c906cc8-094e-49be-b80b-77d196bd50a9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:43.444006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3b7e8-f352-4c35-ac32-ae4f00d0e81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:43.444768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3b7e8-f352-4c35-ac32-ae4f00d0e81c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:43.445048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3b7e8-f352-4c35-ac32-ae4f00d0e81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:43.445321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3b7e8-f352-4c35-ac32-ae4f00d0e81c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:43.446088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3b7e8-f352-4c35-ac32-ae4f00d0e81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:43.447681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3b7e8-f352-4c35-ac32-ae4f00d0e81c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:43.459190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 893/1805] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:43.463235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef78cb-c23a-4eb8-b18a-4759b597a1d2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:43.477970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef78cb-c23a-4eb8-b18a-4759b597a1d2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:43.481179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebec64-ef02-499d-82e7-42d475ce49fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:43.482119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebec64-ef02-499d-82e7-42d475ce49fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:43.482431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebec64-ef02-499d-82e7-42d475ce49fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:43.482714 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebec64-ef02-499d-82e7-42d475ce49fa admin admin] PATH_INFO: `/v3/projects/aed4816595a14ad9a0037172bf2ecfcd/users/6bc8fc0c9cd440ca826a0495b3940010/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:43.498421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebec64-ef02-499d-82e7-42d475ce49fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed4816595a14ad9a0037172bf2ecfcd, user_id=6bc8fc0c9cd440ca826a0495b3940010, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:43.501099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebec64-ef02-499d-82e7-42d475ce49fa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:43.600092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 913/1806] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:43 2022] PUT /identity/v3/projects/aed4816595a14ad9a0037172bf2ecfcd/users/6bc8fc0c9cd440ca826a0495b3940010/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:43.605444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621f7aa-5798-4cf1-a276-3b774446a379 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:43.605741 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621f7aa-5798-4cf1-a276-3b774446a379 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:43.606025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621f7aa-5798-4cf1-a276-3b774446a379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:43.632949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0621f7aa-5798-4cf1-a276-3b774446a379 None None] MFA Rules not processed for user `6bc8fc0c9cd440ca826a0495b3940010`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:43.634540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0621f7aa-5798-4cf1-a276-3b774446a379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:43.696511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 894/1807] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:43.701033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebec64-ef02-499d-82e7-42d475ce49fa admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:43.764695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebec64-ef02-499d-82e7-42d475ce49fa admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:43.765370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:43.770252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276b887-128e-4bc3-9128-a724d46b78ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:43.774352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276b887-128e-4bc3-9128-a724d46b78ed admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:43.774352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276b887-128e-4bc3-9128-a724d46b78ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:43.774352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276b887-128e-4bc3-9128-a724d46b78ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:43.774352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276b887-128e-4bc3-9128-a724d46b78ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:43.774352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276b887-128e-4bc3-9128-a724d46b78ed admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:43.863614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 914/1808] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:43 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:06:43.867948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621f7aa-5798-4cf1-a276-3b774446a379 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:43.880918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621f7aa-5798-4cf1-a276-3b774446a379 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:43.953750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57225fc2-567c-4fb1-b55f-972f28decd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:43.953750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57225fc2-567c-4fb1-b55f-972f28decd6b admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:43.953750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57225fc2-567c-4fb1-b55f-972f28decd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:43.953750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57225fc2-567c-4fb1-b55f-972f28decd6b admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:43.954192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57225fc2-567c-4fb1-b55f-972f28decd6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:43.954192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57225fc2-567c-4fb1-b55f-972f28decd6b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:44.064452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 895/1809] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:43 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:06:44.066866 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276b887-128e-4bc3-9128-a724d46b78ed admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:44.077836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276b887-128e-4bc3-9128-a724d46b78ed admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:44.083340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb637-28b9-45cd-8003-b89ee5fb36d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:44.084437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb637-28b9-45cd-8003-b89ee5fb36d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:44.084437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb637-28b9-45cd-8003-b89ee5fb36d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:44.084740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb637-28b9-45cd-8003-b89ee5fb36d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:44.085224 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28eb637-28b9-45cd-8003-b89ee5fb36d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:44.086889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28eb637-28b9-45cd-8003-b89ee5fb36d0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:44.097566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 915/1810] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:44.102354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57225fc2-567c-4fb1-b55f-972f28decd6b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:44.112493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57225fc2-567c-4fb1-b55f-972f28decd6b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:44.117852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cbe05-6042-45a3-a985-f69cb2fe679b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:44.118026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cbe05-6042-45a3-a985-f69cb2fe679b admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:44.118375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cbe05-6042-45a3-a985-f69cb2fe679b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:44.118648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cbe05-6042-45a3-a985-f69cb2fe679b admin admin] PATH_INFO: `/v3/projects/9fd2cd0b17014208ad0bab1b0ecd91d6/users/992d109e1554474b9a8684fb6265da01/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:44.134008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cbe05-6042-45a3-a985-f69cb2fe679b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd2cd0b17014208ad0bab1b0ecd91d6, user_id=992d109e1554474b9a8684fb6265da01, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:44.136676 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3cbe05-6042-45a3-a985-f69cb2fe679b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:44.239996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 896/1811] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:44 2022] PUT /identity/v3/projects/9fd2cd0b17014208ad0bab1b0ecd91d6/users/992d109e1554474b9a8684fb6265da01/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:44.244178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb637-28b9-45cd-8003-b89ee5fb36d0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:44.254907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb637-28b9-45cd-8003-b89ee5fb36d0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:44.317280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982f737-b76f-465a-a10e-404118d545b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:44.317949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f737-b76f-465a-a10e-404118d545b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:44.318280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f737-b76f-465a-a10e-404118d545b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:44.318555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f737-b76f-465a-a10e-404118d545b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:44.319281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982f737-b76f-465a-a10e-404118d545b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:44.320946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982f737-b76f-465a-a10e-404118d545b6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:44.331317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 916/1812] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:06:44.335737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cbe05-6042-45a3-a985-f69cb2fe679b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:44.347813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cbe05-6042-45a3-a985-f69cb2fe679b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:44.352910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8449957-8678-4700-955c-57603f4ad1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:44.353798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8449957-8678-4700-955c-57603f4ad1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:44.354115 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8449957-8678-4700-955c-57603f4ad1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:44.354395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8449957-8678-4700-955c-57603f4ad1b3 admin admin] PATH_INFO: `/v3/projects/9fd2cd0b17014208ad0bab1b0ecd91d6/users/992d109e1554474b9a8684fb6265da01/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:44.367118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8449957-8678-4700-955c-57603f4ad1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd2cd0b17014208ad0bab1b0ecd91d6, user_id=992d109e1554474b9a8684fb6265da01, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:44.367118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8449957-8678-4700-955c-57603f4ad1b3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:44.371373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 897/1813] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:44 2022] PUT /identity/v3/projects/9fd2cd0b17014208ad0bab1b0ecd91d6/users/992d109e1554474b9a8684fb6265da01/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:44.385234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c90624-cdf0-45ce-be03-2addc9677b9c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:44.385234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c90624-cdf0-45ce-be03-2addc9677b9c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:44.385234 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c90624-cdf0-45ce-be03-2addc9677b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:44.411820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-83c90624-cdf0-45ce-be03-2addc9677b9c None None] MFA Rules not processed for user `992d109e1554474b9a8684fb6265da01`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:44.413110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-83c90624-cdf0-45ce-be03-2addc9677b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:44.504654 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 917/1814] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:44.586548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8449957-8678-4700-955c-57603f4ad1b3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:44.591814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e8449957-8678-4700-955c-57603f4ad1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:44.672884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8449957-8678-4700-955c-57603f4ad1b3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:44.677053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2800957-aee2-4c65-af99-b6ad1060d378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:44.677943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2800957-aee2-4c65-af99-b6ad1060d378 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:44.678391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2800957-aee2-4c65-af99-b6ad1060d378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:44.678860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2800957-aee2-4c65-af99-b6ad1060d378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:44.739030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2800957-aee2-4c65-af99-b6ad1060d378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:44.740508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2800957-aee2-4c65-af99-b6ad1060d378 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:44.749689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 898/1815] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:44.838263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:44.863043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c90624-cdf0-45ce-be03-2addc9677b9c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:44.864490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-83c90624-cdf0-45ce-be03-2addc9677b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:44.938563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c90624-cdf0-45ce-be03-2addc9677b9c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:44.943731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:44.944597 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:44.944878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:44.945153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] PATH_INFO: `/v3/users/d20b89a832d844dcac69081a9ab0222b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:44.946372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20b89a832d844dcac69081a9ab0222b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:44.948331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:45.182678 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20b89a832d844dcac69081a9ab0222b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:45.189910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20b89a832d844dcac69081a9ab0222b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:45.261865 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20b89a832d844dcac69081a9ab0222b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:45.262493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20b89a832d844dcac69081a9ab0222b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:45.263098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20b89a832d844dcac69081a9ab0222b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:45.280932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 918/1816] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:44 2022] DELETE /identity/v3/users/d20b89a832d844dcac69081a9ab0222b => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:45.285839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce69fd2-2405-467e-8bbf-a4827e8aadd9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:45.286442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce69fd2-2405-467e-8bbf-a4827e8aadd9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:45.286587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce69fd2-2405-467e-8bbf-a4827e8aadd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:45.304344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3ce69fd2-2405-467e-8bbf-a4827e8aadd9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:45.305536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3ce69fd2-2405-467e-8bbf-a4827e8aadd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:45.367137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 899/1817] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:45.378672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:45.380171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:45.442115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c3955-a28d-4c0d-8d1e-b2cd9292acba admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:45.445722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a505af-ad71-4329-a40a-993c5457cd54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:45.446406 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a505af-ad71-4329-a40a-993c5457cd54 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:45.446692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a505af-ad71-4329-a40a-993c5457cd54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:45.446970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a505af-ad71-4329-a40a-993c5457cd54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:45.456034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a505af-ad71-4329-a40a-993c5457cd54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:45.464519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a505af-ad71-4329-a40a-993c5457cd54 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:45.469611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 919/1818] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:45.656550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:46.020425 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce69fd2-2405-467e-8bbf-a4827e8aadd9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:46.021793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3ce69fd2-2405-467e-8bbf-a4827e8aadd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:46.034771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce69fd2-2405-467e-8bbf-a4827e8aadd9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:46.042419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:46.043324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:46.046225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:46.046225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] PATH_INFO: `/v3/projects/30ef5f06036840549f823270953ec539` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:46.046225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ef5f06036840549f823270953ec539)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:46.047279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:46.186199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ef5f06036840549f823270953ec539', 'request_id': 'req-63327975-a855-4dd2-b333-8016685d24d6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:46.192123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ef5f06036840549f823270953ec539', 'request_id': 'req-63327975-a855-4dd2-b333-8016685d24d6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:46.265344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ef5f06036840549f823270953ec539', 'request_id': 'req-63327975-a855-4dd2-b333-8016685d24d6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:46.274930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] The token cache is being invalidate because project 30ef5f06036840549f823270953ec539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:46.275218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:46.276006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:46.277655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 900/1819] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:46 2022] DELETE /identity/v3/projects/30ef5f06036840549f823270953ec539 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:46.281586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a505af-ad71-4329-a40a-993c5457cd54 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:46.291840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a505af-ad71-4329-a40a-993c5457cd54 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:46.347755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:46.348703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:46.348990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:46.349258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] PATH_INFO: `/v3/users/6a5f4b270ec24099904f5e626a9a0aee` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:46.351054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a5f4b270ec24099904f5e626a9a0aee)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:46.353208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:46.579254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a5f4b270ec24099904f5e626a9a0aee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:46.586574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a5f4b270ec24099904f5e626a9a0aee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:46.685039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a5f4b270ec24099904f5e626a9a0aee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:46.685655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a5f4b270ec24099904f5e626a9a0aee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:46.686382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a5f4b270ec24099904f5e626a9a0aee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:46.705128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 920/1820] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:46 2022] DELETE /identity/v3/users/6a5f4b270ec24099904f5e626a9a0aee => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:47.166367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:47.169490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:47.181274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63327975-a855-4dd2-b333-8016685d24d6 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:47.223366 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:47.241865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:47.242761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:47.243040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:47.243314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] PATH_INFO: `/v3/projects/f5cb192da07e410cbd80f0a987265b74` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:47.244481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5cb192da07e410cbd80f0a987265b74)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:47.246489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:47.360886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:47.367229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5cb192da07e410cbd80f0a987265b74', 'request_id': 'req-3191c9d7-ee3d-406e-b203-2676c96a6ba3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:47.374078 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5cb192da07e410cbd80f0a987265b74', 'request_id': 'req-3191c9d7-ee3d-406e-b203-2676c96a6ba3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:47.603338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5cb192da07e410cbd80f0a987265b74', 'request_id': 'req-3191c9d7-ee3d-406e-b203-2676c96a6ba3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:47.615090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] The token cache is being invalidate because project f5cb192da07e410cbd80f0a987265b74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:47.615374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:47.616422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:47.618218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 901/1821] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:47 2022] DELETE /identity/v3/projects/f5cb192da07e410cbd80f0a987265b74 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:47.622326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:47.623786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:47.637192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c3ac4-b342-43f4-bac1-ca157e3fc7cb admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:47.696821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:47.697704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:47.697991 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:47.698300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] PATH_INFO: `/v3/users/0df9bb1a720741768e7bfc1a7b30a8a2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:47.699530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0df9bb1a720741768e7bfc1a7b30a8a2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:47.701473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:47.945608 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0df9bb1a720741768e7bfc1a7b30a8a2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:47.952788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0df9bb1a720741768e7bfc1a7b30a8a2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:48.077690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0df9bb1a720741768e7bfc1a7b30a8a2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:48.078705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0df9bb1a720741768e7bfc1a7b30a8a2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:48.080311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e99e64d-928e-4702-9c46-de4707ecb45a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0df9bb1a720741768e7bfc1a7b30a8a2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:48.106630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 921/1822] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:47 2022] DELETE /identity/v3/users/0df9bb1a720741768e7bfc1a7b30a8a2 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:48.612789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:48.746724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:48.750748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:48.781085 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191c9d7-ee3d-406e-b203-2676c96a6ba3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:48.901528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:48.902315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:48.902593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:48.902862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] PATH_INFO: `/v3/projects/3248d63ed2b447aea806395f16188f14` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:48.904059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3248d63ed2b447aea806395f16188f14)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:48.906052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:49.050452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3248d63ed2b447aea806395f16188f14', 'request_id': 'req-d6125ec5-6ca4-4980-824a-6eddfd756591'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:49.060453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3248d63ed2b447aea806395f16188f14', 'request_id': 'req-d6125ec5-6ca4-4980-824a-6eddfd756591'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:49.153817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3248d63ed2b447aea806395f16188f14', 'request_id': 'req-d6125ec5-6ca4-4980-824a-6eddfd756591'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:49.192552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] The token cache is being invalidate because project 3248d63ed2b447aea806395f16188f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:49.192848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:49.194099 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d6125ec5-6ca4-4980-824a-6eddfd756591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:49.199415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 902/1823] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:48 2022] DELETE /identity/v3/projects/3248d63ed2b447aea806395f16188f14 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:49.205651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fea8c-515f-4da5-a173-7440e41dc97b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:49.206257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fea8c-515f-4da5-a173-7440e41dc97b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:49.206257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fea8c-515f-4da5-a173-7440e41dc97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:49.232242 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-298fea8c-515f-4da5-a173-7440e41dc97b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:49.233619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-298fea8c-515f-4da5-a173-7440e41dc97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:49.309324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 922/1824] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:49.311322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e192f-e57e-480a-a0ae-ea8959a0153e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:49.311612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e192f-e57e-480a-a0ae-ea8959a0153e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:49.311890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e192f-e57e-480a-a0ae-ea8959a0153e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:49.337181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-994e192f-e57e-480a-a0ae-ea8959a0153e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:49.338438 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-994e192f-e57e-480a-a0ae-ea8959a0153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:49.350103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 903/1825] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:49.351947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fea8c-515f-4da5-a173-7440e41dc97b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:49.364385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:49.368227 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fea8c-515f-4da5-a173-7440e41dc97b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:49.373018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbdf5d-7bf2-4e5b-b860-2d21c71d3fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:49.373717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbdf5d-7bf2-4e5b-b860-2d21c71d3fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:49.373997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbdf5d-7bf2-4e5b-b860-2d21c71d3fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:49.374402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbdf5d-7bf2-4e5b-b860-2d21c71d3fc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:49.375133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbdf5d-7bf2-4e5b-b860-2d21c71d3fc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:06:49.375439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbdf5d-7bf2-4e5b-b860-2d21c71d3fc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:49.377300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbdf5d-7bf2-4e5b-b860-2d21c71d3fc7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:49.390829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 923/1826] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:06:49.396169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e192f-e57e-480a-a0ae-ea8959a0153e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:49.407747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e192f-e57e-480a-a0ae-ea8959a0153e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:49.413205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315217e1-b505-4009-8f06-c8c00fc62961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:49.413891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315217e1-b505-4009-8f06-c8c00fc62961 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:49.414184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315217e1-b505-4009-8f06-c8c00fc62961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:49.414477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315217e1-b505-4009-8f06-c8c00fc62961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:49.415297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315217e1-b505-4009-8f06-c8c00fc62961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:49.417197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315217e1-b505-4009-8f06-c8c00fc62961 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:49.578541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 904/1827] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:49 2022] POST /identity/v3/projects => generated 401 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:06:49.583821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbdf5d-7bf2-4e5b-b860-2d21c71d3fc7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:49.594291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbdf5d-7bf2-4e5b-b860-2d21c71d3fc7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:49.652621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022f4f8-eb13-4247-b522-2d047462c7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:49.652811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022f4f8-eb13-4247-b522-2d047462c7ea admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:49.653091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022f4f8-eb13-4247-b522-2d047462c7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:49.653367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022f4f8-eb13-4247-b522-2d047462c7ea admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:49.654083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022f4f8-eb13-4247-b522-2d047462c7ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:49.656041 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022f4f8-eb13-4247-b522-2d047462c7ea admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:49.759092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 924/1828] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:49 2022] POST /identity/v3/users => generated 301 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:49.763170 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315217e1-b505-4009-8f06-c8c00fc62961 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:49.774008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315217e1-b505-4009-8f06-c8c00fc62961 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:49.778728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d4fa7-ba02-4031-b5d7-bc4bc5122fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:49.779383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d4fa7-ba02-4031-b5d7-bc4bc5122fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:49.779669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d4fa7-ba02-4031-b5d7-bc4bc5122fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:49.779993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d4fa7-ba02-4031-b5d7-bc4bc5122fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:49.780695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d4fa7-ba02-4031-b5d7-bc4bc5122fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:49.783303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d4fa7-ba02-4031-b5d7-bc4bc5122fb7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:49.793346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 905/1829] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:49.797147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022f4f8-eb13-4247-b522-2d047462c7ea admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:49.807263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022f4f8-eb13-4247-b522-2d047462c7ea admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:49.812113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01b2e2-835f-4d5c-aff0-00b69977a671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:49.813137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01b2e2-835f-4d5c-aff0-00b69977a671 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:49.813369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01b2e2-835f-4d5c-aff0-00b69977a671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:49.813679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01b2e2-835f-4d5c-aff0-00b69977a671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:49.814562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01b2e2-835f-4d5c-aff0-00b69977a671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:49.816407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01b2e2-835f-4d5c-aff0-00b69977a671 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:49.828953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 925/1830] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:49.835774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d4fa7-ba02-4031-b5d7-bc4bc5122fb7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:49.846736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d4fa7-ba02-4031-b5d7-bc4bc5122fb7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:49.851575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6199a3-e7c5-4ded-854d-82da0c936cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:49.852459 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6199a3-e7c5-4ded-854d-82da0c936cce admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:49.852741 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6199a3-e7c5-4ded-854d-82da0c936cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:49.853017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6199a3-e7c5-4ded-854d-82da0c936cce admin admin] PATH_INFO: `/v3/projects/c836a23bb10c4a9aaa1e615076c184e6/users/d732bc54c32d40f7ae74ca902d938f1f/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:49.869602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6199a3-e7c5-4ded-854d-82da0c936cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=c836a23bb10c4a9aaa1e615076c184e6, user_id=d732bc54c32d40f7ae74ca902d938f1f, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:49.872319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6199a3-e7c5-4ded-854d-82da0c936cce admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:49.951866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 906/1831] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:49 2022] PUT /identity/v3/projects/c836a23bb10c4a9aaa1e615076c184e6/users/d732bc54c32d40f7ae74ca902d938f1f/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:49.964092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7608e0-706e-4314-ada6-52ac196c8cfd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:49.964588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7608e0-706e-4314-ada6-52ac196c8cfd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:49.964891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7608e0-706e-4314-ada6-52ac196c8cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:49.994158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9c7608e0-706e-4314-ada6-52ac196c8cfd None None] MFA Rules not processed for user `d732bc54c32d40f7ae74ca902d938f1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:49.995368 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9c7608e0-706e-4314-ada6-52ac196c8cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:50.067239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 926/1832] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:50.073511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6199a3-e7c5-4ded-854d-82da0c936cce admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:50.135954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6199a3-e7c5-4ded-854d-82da0c936cce admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:50.140828 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cd7c0-80dc-411d-82fa-7217e8909bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:50.141504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cd7c0-80dc-411d-82fa-7217e8909bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:50.142655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cd7c0-80dc-411d-82fa-7217e8909bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:50.142969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cd7c0-80dc-411d-82fa-7217e8909bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:50.143672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cd7c0-80dc-411d-82fa-7217e8909bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:50.145611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cd7c0-80dc-411d-82fa-7217e8909bd8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:50.266248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 907/1833] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:50 2022] POST /identity/v3/projects => generated 401 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:06:50.270752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7608e0-706e-4314-ada6-52ac196c8cfd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:50.281900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7608e0-706e-4314-ada6-52ac196c8cfd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:50.341316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38d9fc-f363-45eb-aea8-a188839885ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:50.342004 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38d9fc-f363-45eb-aea8-a188839885ce admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:50.342306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38d9fc-f363-45eb-aea8-a188839885ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:50.342678 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38d9fc-f363-45eb-aea8-a188839885ce admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:50.343265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38d9fc-f363-45eb-aea8-a188839885ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:50.345223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38d9fc-f363-45eb-aea8-a188839885ce admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:50.450680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 927/1834] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:50 2022] POST /identity/v3/users => generated 307 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:50.454773 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cd7c0-80dc-411d-82fa-7217e8909bd8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:50.464930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cd7c0-80dc-411d-82fa-7217e8909bd8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:50.469528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bada8-fd93-452b-9159-02f9e63a67b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:50.470496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bada8-fd93-452b-9159-02f9e63a67b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:50.470780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bada8-fd93-452b-9159-02f9e63a67b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:50.471051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bada8-fd93-452b-9159-02f9e63a67b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:50.472422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bada8-fd93-452b-9159-02f9e63a67b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:50.473548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bada8-fd93-452b-9159-02f9e63a67b6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:50.480695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:50.486898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 908/1835] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:50.491361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38d9fc-f363-45eb-aea8-a188839885ce admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:50.501861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38d9fc-f363-45eb-aea8-a188839885ce admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:50.507138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180646a-59b0-4ca2-a23d-be798bd3b68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:50.508047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180646a-59b0-4ca2-a23d-be798bd3b68f admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:50.508333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180646a-59b0-4ca2-a23d-be798bd3b68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:50.508611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180646a-59b0-4ca2-a23d-be798bd3b68f admin admin] PATH_INFO: `/v3/projects/6163149bce154604b6a9a4ff3ed95c8f/users/1f4538e72fd44b87bf2dc220cd906783/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:50.524224 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180646a-59b0-4ca2-a23d-be798bd3b68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6163149bce154604b6a9a4ff3ed95c8f, user_id=1f4538e72fd44b87bf2dc220cd906783, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:50.527053 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180646a-59b0-4ca2-a23d-be798bd3b68f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:50.622690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 928/1836] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:50 2022] PUT /identity/v3/projects/6163149bce154604b6a9a4ff3ed95c8f/users/1f4538e72fd44b87bf2dc220cd906783/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:50.627702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bada8-fd93-452b-9159-02f9e63a67b6 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:50.637568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bada8-fd93-452b-9159-02f9e63a67b6 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:50.689540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:50.693613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a947e0-30bb-433e-9901-007ebbfc1dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:50.694409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a947e0-30bb-433e-9901-007ebbfc1dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:50.694688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a947e0-30bb-433e-9901-007ebbfc1dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:50.694983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a947e0-30bb-433e-9901-007ebbfc1dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:50.698022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a947e0-30bb-433e-9901-007ebbfc1dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:50.698022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a947e0-30bb-433e-9901-007ebbfc1dd3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:50.707632 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 909/1837] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:50.711987 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180646a-59b0-4ca2-a23d-be798bd3b68f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:50.722773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180646a-59b0-4ca2-a23d-be798bd3b68f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:50.727377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5912eb6-7b72-4fcb-9184-7751d0389b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:50.728267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5912eb6-7b72-4fcb-9184-7751d0389b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:50.728541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5912eb6-7b72-4fcb-9184-7751d0389b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:50.728811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5912eb6-7b72-4fcb-9184-7751d0389b12 admin admin] PATH_INFO: `/v3/projects/6163149bce154604b6a9a4ff3ed95c8f/users/1f4538e72fd44b87bf2dc220cd906783/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:50.730743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5912eb6-7b72-4fcb-9184-7751d0389b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6163149bce154604b6a9a4ff3ed95c8f, user_id=1f4538e72fd44b87bf2dc220cd906783, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:50.733423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5912eb6-7b72-4fcb-9184-7751d0389b12 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:50.744423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 929/1838] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:50 2022] PUT /identity/v3/projects/6163149bce154604b6a9a4ff3ed95c8f/users/1f4538e72fd44b87bf2dc220cd906783/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:50.755672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bbf49-d776-49a2-bfe7-21d51fcf4d36 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:50.755960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bbf49-d776-49a2-bfe7-21d51fcf4d36 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:50.756235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bbf49-d776-49a2-bfe7-21d51fcf4d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:50.783289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-024bbf49-d776-49a2-bfe7-21d51fcf4d36 None None] MFA Rules not processed for user `1f4538e72fd44b87bf2dc220cd906783`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:50.784493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-024bbf49-d776-49a2-bfe7-21d51fcf4d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:50.853023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 910/1839] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:50 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:50.865208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5912eb6-7b72-4fcb-9184-7751d0389b12 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:50.867320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d5912eb6-7b72-4fcb-9184-7751d0389b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:50.924215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5912eb6-7b72-4fcb-9184-7751d0389b12 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:50.927941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b5458-8d68-4dac-936a-c969c76264f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:50.928586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b5458-8d68-4dac-936a-c969c76264f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:50.928871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b5458-8d68-4dac-936a-c969c76264f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:50.929145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b5458-8d68-4dac-936a-c969c76264f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:50.938174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b5458-8d68-4dac-936a-c969c76264f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:50.939668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b5458-8d68-4dac-936a-c969c76264f4 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:50.949085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 930/1840] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:50 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:51.185437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bbf49-d776-49a2-bfe7-21d51fcf4d36 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:51.201240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bbf49-d776-49a2-bfe7-21d51fcf4d36 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:51.206432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c509503-7bbc-4d92-bff9-ed25818ee241 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:51.206432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c509503-7bbc-4d92-bff9-ed25818ee241 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:51.206671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c509503-7bbc-4d92-bff9-ed25818ee241 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:51.206991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c509503-7bbc-4d92-bff9-ed25818ee241 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:51.291384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c509503-7bbc-4d92-bff9-ed25818ee241 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:51.293824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c509503-7bbc-4d92-bff9-ed25818ee241 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:51.310354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 911/1841] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:51.628025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:52.084985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:52.420636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b5458-8d68-4dac-936a-c969c76264f4 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:52.422021 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a22b5458-8d68-4dac-936a-c969c76264f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:52.443014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b5458-8d68-4dac-936a-c969c76264f4 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:52.443014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e31f1-a362-4eb6-be00-4f2bd5d1bd96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:52.443014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e31f1-a362-4eb6-be00-4f2bd5d1bd96 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:52.443014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e31f1-a362-4eb6-be00-4f2bd5d1bd96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:52.443413 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e31f1-a362-4eb6-be00-4f2bd5d1bd96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:52.443413 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-891e31f1-a362-4eb6-be00-4f2bd5d1bd96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:52.509437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e31f1-a362-4eb6-be00-4f2bd5d1bd96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:52.510910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e31f1-a362-4eb6-be00-4f2bd5d1bd96 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:52.521820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 931/1842] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:53.026960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:53.035589 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1b21a-dd5a-4aae-a493-72bdd30c1e19 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:53.036760 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1b21a-dd5a-4aae-a493-72bdd30c1e19 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:53.037056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1b21a-dd5a-4aae-a493-72bdd30c1e19 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:53.038849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 912/1843] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 09:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:06:53.045191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748b1b-4c71-4370-a3ca-74391dacc3ab None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:53.045481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748b1b-4c71-4370-a3ca-74391dacc3ab None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:53.045759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748b1b-4c71-4370-a3ca-74391dacc3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:53.072828 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1b21a-dd5a-4aae-a493-72bdd30c1e19 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:53.075588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d1e1b21a-dd5a-4aae-a493-72bdd30c1e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:53.091418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0e748b1b-4c71-4370-a3ca-74391dacc3ab None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:53.092546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0e748b1b-4c71-4370-a3ca-74391dacc3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:53.188784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 932/1844] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 09:06:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:53.221993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e748b1b-4c71-4370-a3ca-74391dacc3ab None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:53.223732 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0e748b1b-4c71-4370-a3ca-74391dacc3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:53.237111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1b21a-dd5a-4aae-a493-72bdd30c1e19 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:53.242064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e748b1b-4c71-4370-a3ca-74391dacc3ab None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:53.245022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:53.249743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfe3c4-16cf-4413-90f8-d26e49e3d206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:53.250129 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:53.250623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfe3c4-16cf-4413-90f8-d26e49e3d206 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:53.251000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:53.251126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfe3c4-16cf-4413-90f8-d26e49e3d206 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:53.251362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfe3c4-16cf-4413-90f8-d26e49e3d206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:53.251783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] PATH_INFO: `/v3/users/41311259448942d896d3c85715a67d46` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:53.253735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] RBAC: Authorizing `identity:delete_user(user_id=41311259448942d896d3c85715a67d46)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:53.256006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:53.271229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfe3c4-16cf-4413-90f8-d26e49e3d206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:53.272912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfe3c4-16cf-4413-90f8-d26e49e3d206 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:53.288300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 933/1845] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:06:53.526380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41311259448942d896d3c85715a67d46'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:53.536100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41311259448942d896d3c85715a67d46'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:53.548122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41311259448942d896d3c85715a67d46'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:53.605150 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:53.645847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41311259448942d896d3c85715a67d46'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:53.646849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41311259448942d896d3c85715a67d46'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:53.663625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 913/1846] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:53 2022] DELETE /identity/v3/users/41311259448942d896d3c85715a67d46 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:53.668343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db7887-4722-4900-af2f-30653a6d2c62 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:53.668630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db7887-4722-4900-af2f-30653a6d2c62 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:53.669175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db7887-4722-4900-af2f-30653a6d2c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:53.703111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b3db7887-4722-4900-af2f-30653a6d2c62 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:53.705198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b3db7887-4722-4900-af2f-30653a6d2c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:53.796448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 934/1847] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:53.811145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:53.813405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:53.844053 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db7887-4722-4900-af2f-30653a6d2c62 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:53.852870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b3db7887-4722-4900-af2f-30653a6d2c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:53.879202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db7887-4722-4900-af2f-30653a6d2c62 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:53.890519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:53.891866 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:53.892211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:53.892496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] PATH_INFO: `/v3/users/542c321896f94802b0acf9be1e6d138b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:53.894286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=542c321896f94802b0acf9be1e6d138b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:53.896773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:53.918848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866dedbf-acbf-4eeb-982a-7e758f01a05a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:53.924947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb0f3c-a850-4c71-aa12-4b875e8ac766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:53.926166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb0f3c-a850-4c71-aa12-4b875e8ac766 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:53.927734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb0f3c-a850-4c71-aa12-4b875e8ac766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:53.928235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb0f3c-a850-4c71-aa12-4b875e8ac766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:53.946225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb0f3c-a850-4c71-aa12-4b875e8ac766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:53.948019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb0f3c-a850-4c71-aa12-4b875e8ac766 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:53.961177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 914/1848] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:54.051336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:54.114723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542c321896f94802b0acf9be1e6d138b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:54.122214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542c321896f94802b0acf9be1e6d138b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:54.193424 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:54.222155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542c321896f94802b0acf9be1e6d138b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:54.222794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542c321896f94802b0acf9be1e6d138b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:54.223476 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542c321896f94802b0acf9be1e6d138b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:54.243622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 935/1849] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:53 2022] DELETE /identity/v3/users/542c321896f94802b0acf9be1e6d138b => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:54.258282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b121dd-9d03-4a40-be42-d7a72334e9e0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:54.258282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b121dd-9d03-4a40-be42-d7a72334e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:54.258282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b121dd-9d03-4a40-be42-d7a72334e9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:54.286265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c4b121dd-9d03-4a40-be42-d7a72334e9e0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:54.287765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c4b121dd-9d03-4a40-be42-d7a72334e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:54.354816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 915/1850] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:54.362814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:54.364564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:54.427433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f13e8-39ad-4b6c-b2b7-ae224aa3e535 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:54.431282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfaec8-3700-44e1-a1f4-bf49ffd6857a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:54.431996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfaec8-3700-44e1-a1f4-bf49ffd6857a service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:54.432345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfaec8-3700-44e1-a1f4-bf49ffd6857a service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:54.432627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfaec8-3700-44e1-a1f4-bf49ffd6857a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:54.443679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfaec8-3700-44e1-a1f4-bf49ffd6857a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:54.445091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfaec8-3700-44e1-a1f4-bf49ffd6857a service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:54.464626 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 936/1851] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:54.507366 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b121dd-9d03-4a40-be42-d7a72334e9e0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:54.508785 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c4b121dd-9d03-4a40-be42-d7a72334e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:54.521977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b121dd-9d03-4a40-be42-d7a72334e9e0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:54.526877 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:54.527757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:54.528038 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:54.528311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] PATH_INFO: `/v3/projects/27a5e75617e749908a9fdf29f91ec3e2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:54.529808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27a5e75617e749908a9fdf29f91ec3e2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:54.531764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:54.676887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27a5e75617e749908a9fdf29f91ec3e2', 'request_id': 'req-d848a4ad-5edc-47c4-9771-c0f0ab27a144'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:54.684323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27a5e75617e749908a9fdf29f91ec3e2', 'request_id': 'req-d848a4ad-5edc-47c4-9771-c0f0ab27a144'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:54.781671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27a5e75617e749908a9fdf29f91ec3e2', 'request_id': 'req-d848a4ad-5edc-47c4-9771-c0f0ab27a144'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:54.794798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] The token cache is being invalidate because project 27a5e75617e749908a9fdf29f91ec3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:54.795115 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:54.795799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:54.797400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 916/1852] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:54 2022] DELETE /identity/v3/projects/27a5e75617e749908a9fdf29f91ec3e2 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:54.801660 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfaec8-3700-44e1-a1f4-bf49ffd6857a service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:54.811943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfaec8-3700-44e1-a1f4-bf49ffd6857a service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:54.871950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:54.872830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:54.873114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:54.873393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] PATH_INFO: `/v3/users/342cd72c66ed477580ebe746b82d5193` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:54.874644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=342cd72c66ed477580ebe746b82d5193)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:54.876791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:54.937116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:55.091085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342cd72c66ed477580ebe746b82d5193'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.100983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342cd72c66ed477580ebe746b82d5193'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.164499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:55.166284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:55.183917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342cd72c66ed477580ebe746b82d5193'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.184691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848a4ad-5edc-47c4-9771-c0f0ab27a144 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:55.184779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342cd72c66ed477580ebe746b82d5193'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.184779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342cd72c66ed477580ebe746b82d5193'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.184935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:55.191501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:55.192447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:55.192744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:55.193054 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] PATH_INFO: `/v3/projects/04254b9b7bdc426382c0232f5aedd038` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:55.194287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04254b9b7bdc426382c0232f5aedd038)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:55.196391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:55.201595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 937/1853] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:54 2022] DELETE /identity/v3/users/342cd72c66ed477580ebe746b82d5193 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:55.307995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04254b9b7bdc426382c0232f5aedd038', 'request_id': 'req-52c0d926-c6b7-4ff0-bafb-e9db213a968d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.314072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04254b9b7bdc426382c0232f5aedd038', 'request_id': 'req-52c0d926-c6b7-4ff0-bafb-e9db213a968d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.437618 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04254b9b7bdc426382c0232f5aedd038', 'request_id': 'req-52c0d926-c6b7-4ff0-bafb-e9db213a968d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.450689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] The token cache is being invalidate because project 04254b9b7bdc426382c0232f5aedd038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:55.451035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.451763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.453359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 917/1854] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:55 2022] DELETE /identity/v3/projects/04254b9b7bdc426382c0232f5aedd038 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:55.457411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:55.459594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:55.559755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c77993-8e56-49c7-9929-8ccbf827c904 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:55.624378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:55.625259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:55.626133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:55.626569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] PATH_INFO: `/v3/users/e65383f5aa0d4dc39c7eb14a51a4e1ca` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:55.628199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e65383f5aa0d4dc39c7eb14a51a4e1ca)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:55.630548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:55.720913 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:55.722934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:55.737716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0d926-c6b7-4ff0-bafb-e9db213a968d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:55.742543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:55.743649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:55.743922 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:55.744317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] PATH_INFO: `/v3/projects/68960387660f4d29abe165d68f77b90f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:55.745907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=68960387660f4d29abe165d68f77b90f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:55.748289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:55.795295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e65383f5aa0d4dc39c7eb14a51a4e1ca'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.802116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e65383f5aa0d4dc39c7eb14a51a4e1ca'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.902674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68960387660f4d29abe165d68f77b90f', 'request_id': 'req-6a2f03da-d65e-44ea-ab82-857587bddc7c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.908611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68960387660f4d29abe165d68f77b90f', 'request_id': 'req-6a2f03da-d65e-44ea-ab82-857587bddc7c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.912630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e65383f5aa0d4dc39c7eb14a51a4e1ca'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.913489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e65383f5aa0d4dc39c7eb14a51a4e1ca'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.914145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bc464683-10ec-4cbb-b546-eba4b7b89545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e65383f5aa0d4dc39c7eb14a51a4e1ca'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:55.932271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 938/1855] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:55 2022] DELETE /identity/v3/users/e65383f5aa0d4dc39c7eb14a51a4e1ca => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:56.027930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68960387660f4d29abe165d68f77b90f', 'request_id': 'req-6a2f03da-d65e-44ea-ab82-857587bddc7c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:56.047711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] The token cache is being invalidate because project 68960387660f4d29abe165d68f77b90f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:56.048060 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:56.048753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6a2f03da-d65e-44ea-ab82-857587bddc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:56.050586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 918/1856] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:55 2022] DELETE /identity/v3/projects/68960387660f4d29abe165d68f77b90f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:56.056577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3772a-605c-4769-9bb0-91cf0ce114ed None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.057524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3772a-605c-4769-9bb0-91cf0ce114ed None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.057812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3772a-605c-4769-9bb0-91cf0ce114ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.076039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fdb3772a-605c-4769-9bb0-91cf0ce114ed None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:56.077249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fdb3772a-605c-4769-9bb0-91cf0ce114ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:56.090605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:56.140714 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 939/1857] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:56.151880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866eb9c-adea-4cec-9ec6-cb3161f986f3 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.152164 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866eb9c-adea-4cec-9ec6-cb3161f986f3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.152465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866eb9c-adea-4cec-9ec6-cb3161f986f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.194776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3866eb9c-adea-4cec-9ec6-cb3161f986f3 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:56.196209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3866eb9c-adea-4cec-9ec6-cb3161f986f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:56.210411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 919/1858] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:56.214613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3772a-605c-4769-9bb0-91cf0ce114ed None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:56.224967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3772a-605c-4769-9bb0-91cf0ce114ed None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:56.229908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee725ac6-9d17-4faa-a222-5d12c5ac0cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:56.230686 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee725ac6-9d17-4faa-a222-5d12c5ac0cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.231079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee725ac6-9d17-4faa-a222-5d12c5ac0cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.231362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee725ac6-9d17-4faa-a222-5d12c5ac0cd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.232305 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee725ac6-9d17-4faa-a222-5d12c5ac0cd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:06:56.232627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee725ac6-9d17-4faa-a222-5d12c5ac0cd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:56.234341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee725ac6-9d17-4faa-a222-5d12c5ac0cd2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:56.238729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:56.256360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 940/1859] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:06:56.257939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866eb9c-adea-4cec-9ec6-cb3161f986f3 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:56.270164 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866eb9c-adea-4cec-9ec6-cb3161f986f3 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:56.275564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266c05c-8135-4a7f-ad38-41e3d54d6202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:56.276259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266c05c-8135-4a7f-ad38-41e3d54d6202 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.276541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266c05c-8135-4a7f-ad38-41e3d54d6202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.276817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266c05c-8135-4a7f-ad38-41e3d54d6202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.277549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266c05c-8135-4a7f-ad38-41e3d54d6202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:56.279662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266c05c-8135-4a7f-ad38-41e3d54d6202 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:56.358737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 920/1860] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:56 2022] POST /identity/v3/projects => generated 407 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:56.363594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee725ac6-9d17-4faa-a222-5d12c5ac0cd2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:56.373400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee725ac6-9d17-4faa-a222-5d12c5ac0cd2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:56.392802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266c05c-8135-4a7f-ad38-41e3d54d6202 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:56.394776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5266c05c-8135-4a7f-ad38-41e3d54d6202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:56.445859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ddc9a-fb24-48fc-b5e9-238d95445b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:56.446260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ddc9a-fb24-48fc-b5e9-238d95445b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.446594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ddc9a-fb24-48fc-b5e9-238d95445b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.446998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ddc9a-fb24-48fc-b5e9-238d95445b1d admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.447978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ddc9a-fb24-48fc-b5e9-238d95445b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:56.450071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ddc9a-fb24-48fc-b5e9-238d95445b1d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:56.474480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266c05c-8135-4a7f-ad38-41e3d54d6202 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:56.479231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:56.480039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.480323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.480660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] PATH_INFO: `/v3/projects/c9318989c1864a25b8aea5c99d179931` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.481916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9318989c1864a25b8aea5c99d179931)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:56.483969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:56.577984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 941/1861] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:56 2022] POST /identity/v3/users => generated 304 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:56.579924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ddc9a-fb24-48fc-b5e9-238d95445b1d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:56.591208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ddc9a-fb24-48fc-b5e9-238d95445b1d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:56.596969 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26841f-331b-40ca-b17c-791139560306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:56.597647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26841f-331b-40ca-b17c-791139560306 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.597955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26841f-331b-40ca-b17c-791139560306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.604321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26841f-331b-40ca-b17c-791139560306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.605117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26841f-331b-40ca-b17c-791139560306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:56.606796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26841f-331b-40ca-b17c-791139560306 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:56.620202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 942/1862] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:56.621865 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26841f-331b-40ca-b17c-791139560306 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:56.632740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26841f-331b-40ca-b17c-791139560306 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:56.637306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf930a-d2f0-41c3-837c-9d7783ea2dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:56.637949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf930a-d2f0-41c3-837c-9d7783ea2dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.638290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf930a-d2f0-41c3-837c-9d7783ea2dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.640960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf930a-d2f0-41c3-837c-9d7783ea2dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.640960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf930a-d2f0-41c3-837c-9d7783ea2dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:56.640960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf930a-d2f0-41c3-837c-9d7783ea2dc5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:56.652146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 943/1863] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:06:56.656089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf930a-d2f0-41c3-837c-9d7783ea2dc5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:56.666049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf930a-d2f0-41c3-837c-9d7783ea2dc5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:56.689849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9318989c1864a25b8aea5c99d179931', 'request_id': 'req-78a40bef-bce4-4a89-ac86-9ca6a5970235'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:56.696849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9318989c1864a25b8aea5c99d179931', 'request_id': 'req-78a40bef-bce4-4a89-ac86-9ca6a5970235'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:56.739786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1db09-76f7-4f62-83e1-cd72c457cf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:56.741675 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1db09-76f7-4f62-83e1-cd72c457cf66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.741952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1db09-76f7-4f62-83e1-cd72c457cf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.742252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1db09-76f7-4f62-83e1-cd72c457cf66 admin admin] PATH_INFO: `/v3/projects/bdc3e6f1740445f2884bafb487d27906/users/f824a504d04c4abcb9360db5a5962a5b/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.761672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1db09-76f7-4f62-83e1-cd72c457cf66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc3e6f1740445f2884bafb487d27906, user_id=f824a504d04c4abcb9360db5a5962a5b, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:56.764792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1db09-76f7-4f62-83e1-cd72c457cf66 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:56.796979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9318989c1864a25b8aea5c99d179931', 'request_id': 'req-78a40bef-bce4-4a89-ac86-9ca6a5970235'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:56.817510 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] The token cache is being invalidate because project c9318989c1864a25b8aea5c99d179931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:56.818061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:56.818822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-78a40bef-bce4-4a89-ac86-9ca6a5970235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:56.820600 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 921/1864] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:56 2022] DELETE /identity/v3/projects/c9318989c1864a25b8aea5c99d179931 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:56.825327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26442120-fd65-40d7-885a-93c6f1a1b091 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.825920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26442120-fd65-40d7-885a-93c6f1a1b091 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.826242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26442120-fd65-40d7-885a-93c6f1a1b091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.850206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-26442120-fd65-40d7-885a-93c6f1a1b091 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:56.850338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-26442120-fd65-40d7-885a-93c6f1a1b091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:56.858999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 922/1865] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:56.867303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d4878-2d1a-45bb-b9e9-421c5ce9dba7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.868084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d4878-2d1a-45bb-b9e9-421c5ce9dba7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.868194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d4878-2d1a-45bb-b9e9-421c5ce9dba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.870500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 944/1866] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:56 2022] PUT /identity/v3/projects/bdc3e6f1740445f2884bafb487d27906/users/f824a504d04c4abcb9360db5a5962a5b/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:56.874486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4fe3-b17a-4f70-a36d-c8649d94eb27 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.874884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4fe3-b17a-4f70-a36d-c8649d94eb27 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.875282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4fe3-b17a-4f70-a36d-c8649d94eb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.888168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7a8d4878-2d1a-45bb-b9e9-421c5ce9dba7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:56.890324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7a8d4878-2d1a-45bb-b9e9-421c5ce9dba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:56.930169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-08ef4fe3-b17a-4f70-a36d-c8649d94eb27 None None] MFA Rules not processed for user `f824a504d04c4abcb9360db5a5962a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:56.931520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-08ef4fe3-b17a-4f70-a36d-c8649d94eb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:56.968019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 923/1867] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:56.972318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d4878-2d1a-45bb-b9e9-421c5ce9dba7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:56.982465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d4878-2d1a-45bb-b9e9-421c5ce9dba7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:56.988916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd47c7-9bae-4e72-bec1-3f8ffb9685a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:56.994490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd47c7-9bae-4e72-bec1-3f8ffb9685a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:56.994490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd47c7-9bae-4e72-bec1-3f8ffb9685a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:56.994490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd47c7-9bae-4e72-bec1-3f8ffb9685a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:56.994490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd47c7-9bae-4e72-bec1-3f8ffb9685a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:06:56.994490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd47c7-9bae-4e72-bec1-3f8ffb9685a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:56.994490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd47c7-9bae-4e72-bec1-3f8ffb9685a7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.012044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 945/1868] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:57.012232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 924/1869] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:06:57.014848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef4fe3-b17a-4f70-a36d-c8649d94eb27 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.015100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd47c7-9bae-4e72-bec1-3f8ffb9685a7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.016661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-08ef4fe3-b17a-4f70-a36d-c8649d94eb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:57.028505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd47c7-9bae-4e72-bec1-3f8ffb9685a7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.034841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd126c-1ee3-4177-9b7b-e24690c180f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.035551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd126c-1ee3-4177-9b7b-e24690c180f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.035860 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd126c-1ee3-4177-9b7b-e24690c180f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.035977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef4fe3-b17a-4f70-a36d-c8649d94eb27 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.036158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd126c-1ee3-4177-9b7b-e24690c180f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.036909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd126c-1ee3-4177-9b7b-e24690c180f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.038929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd126c-1ee3-4177-9b7b-e24690c180f6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.042096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75231a-3656-4726-bffa-b299290bd9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.043543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75231a-3656-4726-bffa-b299290bd9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.045735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75231a-3656-4726-bffa-b299290bd9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.046010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75231a-3656-4726-bffa-b299290bd9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.046938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75231a-3656-4726-bffa-b299290bd9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.049323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75231a-3656-4726-bffa-b299290bd9c0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.166243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 946/1870] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:57 2022] POST /identity/v3/projects => generated 407 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:57.170786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75231a-3656-4726-bffa-b299290bd9c0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.171356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 925/1871] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:57 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:57.180762 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd126c-1ee3-4177-9b7b-e24690c180f6 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.240211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75231a-3656-4726-bffa-b299290bd9c0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.264544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:57.277543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd126c-1ee3-4177-9b7b-e24690c180f6 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.291723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585d15d-7abd-492b-bece-a60c249774d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.292575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585d15d-7abd-492b-bece-a60c249774d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.292966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585d15d-7abd-492b-bece-a60c249774d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.293357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585d15d-7abd-492b-bece-a60c249774d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.294474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585d15d-7abd-492b-bece-a60c249774d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.298861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585d15d-7abd-492b-bece-a60c249774d0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.306482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b9784-7946-4066-b66b-739e1b68080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.307580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b9784-7946-4066-b66b-739e1b68080f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.317495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b9784-7946-4066-b66b-739e1b68080f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.317495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b9784-7946-4066-b66b-739e1b68080f admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.317495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b9784-7946-4066-b66b-739e1b68080f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.317495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b9784-7946-4066-b66b-739e1b68080f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.340290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:57.400546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 926/1872] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:57 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:06:57.402677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 947/1873] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:57 2022] POST /identity/v3/users => generated 310 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:06:57.404287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585d15d-7abd-492b-bece-a60c249774d0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.407328 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b9784-7946-4066-b66b-739e1b68080f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.415309 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585d15d-7abd-492b-bece-a60c249774d0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.418801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b9784-7946-4066-b66b-739e1b68080f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.423281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9b9ef-45d6-4bfd-91b6-3079371029dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.424331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9b9ef-45d6-4bfd-91b6-3079371029dc admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.424715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9b9ef-45d6-4bfd-91b6-3079371029dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.425066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9b9ef-45d6-4bfd-91b6-3079371029dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.425857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc092f-ae16-4562-8936-d3a538ff71bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.426055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9b9ef-45d6-4bfd-91b6-3079371029dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.426392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc092f-ae16-4562-8936-d3a538ff71bc admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.426734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc092f-ae16-4562-8936-d3a538ff71bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.427071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc092f-ae16-4562-8936-d3a538ff71bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.427878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc092f-ae16-4562-8936-d3a538ff71bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.428117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9b9ef-45d6-4bfd-91b6-3079371029dc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.429738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc092f-ae16-4562-8936-d3a538ff71bc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.442096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 948/1874] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:57.443456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 927/1875] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:57.445000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc092f-ae16-4562-8936-d3a538ff71bc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.445936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9b9ef-45d6-4bfd-91b6-3079371029dc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.456898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc092f-ae16-4562-8936-d3a538ff71bc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.459177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9b9ef-45d6-4bfd-91b6-3079371029dc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.467857 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11639af2-187a-4afc-9b9c-a4e14af7292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.468545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11639af2-187a-4afc-9b9c-a4e14af7292e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.468830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11639af2-187a-4afc-9b9c-a4e14af7292e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.470358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bdf56-4bff-48b8-b2e0-192441b84c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.471348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bdf56-4bff-48b8-b2e0-192441b84c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.471673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bdf56-4bff-48b8-b2e0-192441b84c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.471888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bdf56-4bff-48b8-b2e0-192441b84c8e admin admin] PATH_INFO: `/v3/projects/b8c94d2924364e189aa4f1b989008f74/users/aafddfa737124229b77dd87fdd3d6c22/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.472876 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11639af2-187a-4afc-9b9c-a4e14af7292e admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.473621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11639af2-187a-4afc-9b9c-a4e14af7292e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.475263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11639af2-187a-4afc-9b9c-a4e14af7292e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.488707 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 949/1876] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:57.493363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11639af2-187a-4afc-9b9c-a4e14af7292e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.504825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11639af2-187a-4afc-9b9c-a4e14af7292e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.505364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bdf56-4bff-48b8-b2e0-192441b84c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c94d2924364e189aa4f1b989008f74, user_id=aafddfa737124229b77dd87fdd3d6c22, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.510174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bdf56-4bff-48b8-b2e0-192441b84c8e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.510640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c392d4e-5f52-43ff-83cf-81df3b23b7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.512045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c392d4e-5f52-43ff-83cf-81df3b23b7a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.512786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c392d4e-5f52-43ff-83cf-81df3b23b7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.512786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c392d4e-5f52-43ff-83cf-81df3b23b7a9 admin admin] PATH_INFO: `/v3/projects/746c0db8002347f48bd3c203c6616ed8/users/2fdaf317517b4b3cb097be5aa00ab5c0/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.529741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c392d4e-5f52-43ff-83cf-81df3b23b7a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=746c0db8002347f48bd3c203c6616ed8, user_id=2fdaf317517b4b3cb097be5aa00ab5c0, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.535649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c392d4e-5f52-43ff-83cf-81df3b23b7a9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.593218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 928/1877] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:57 2022] PUT /identity/v3/projects/b8c94d2924364e189aa4f1b989008f74/users/aafddfa737124229b77dd87fdd3d6c22/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:57.604694 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bdf56-4bff-48b8-b2e0-192441b84c8e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.627450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 950/1878] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:57 2022] PUT /identity/v3/projects/746c0db8002347f48bd3c203c6616ed8/users/2fdaf317517b4b3cb097be5aa00ab5c0/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:57.643369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bddb70-1dfd-4f20-a5f8-302a770869cd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.643944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bddb70-1dfd-4f20-a5f8-302a770869cd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.644822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bddb70-1dfd-4f20-a5f8-302a770869cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.674208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c2bddb70-1dfd-4f20-a5f8-302a770869cd None None] MFA Rules not processed for user `2fdaf317517b4b3cb097be5aa00ab5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:57.675309 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bdf56-4bff-48b8-b2e0-192441b84c8e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.675438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c2bddb70-1dfd-4f20-a5f8-302a770869cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:57.752840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84826ac2-20cb-4e8b-abb6-893e0f690686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.753525 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84826ac2-20cb-4e8b-abb6-893e0f690686 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.753810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84826ac2-20cb-4e8b-abb6-893e0f690686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.754313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84826ac2-20cb-4e8b-abb6-893e0f690686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.755704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84826ac2-20cb-4e8b-abb6-893e0f690686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.757621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84826ac2-20cb-4e8b-abb6-893e0f690686 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.767466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 951/1879] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:57.775777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 929/1880] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:06:57.781028 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84826ac2-20cb-4e8b-abb6-893e0f690686 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.781188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bddb70-1dfd-4f20-a5f8-302a770869cd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.796686 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84826ac2-20cb-4e8b-abb6-893e0f690686 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.805379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786b72a-028d-4d9e-b4e2-706003c45feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.805379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786b72a-028d-4d9e-b4e2-706003c45feb admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.805573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786b72a-028d-4d9e-b4e2-706003c45feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.805644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bddb70-1dfd-4f20-a5f8-302a770869cd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:57.805724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786b72a-028d-4d9e-b4e2-706003c45feb admin admin] PATH_INFO: `/v3/projects/b8c94d2924364e189aa4f1b989008f74/users/aafddfa737124229b77dd87fdd3d6c22/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.807772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786b72a-028d-4d9e-b4e2-706003c45feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c94d2924364e189aa4f1b989008f74, user_id=aafddfa737124229b77dd87fdd3d6c22, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.810478 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786b72a-028d-4d9e-b4e2-706003c45feb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.810590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9e75d-6727-49de-b466-cc34194aedb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:57.811531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9e75d-6727-49de-b466-cc34194aedb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.811777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9e75d-6727-49de-b466-cc34194aedb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.812109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9e75d-6727-49de-b466-cc34194aedb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.812808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9e75d-6727-49de-b466-cc34194aedb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:57.814985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9e75d-6727-49de-b466-cc34194aedb7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:57.827440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 930/1881] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:57 2022] PUT /identity/v3/projects/b8c94d2924364e189aa4f1b989008f74/users/aafddfa737124229b77dd87fdd3d6c22/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:57.838702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0512ac7-9427-4e9c-b7f7-97fe10d6d52a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:57.839152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0512ac7-9427-4e9c-b7f7-97fe10d6d52a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:57.841422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0512ac7-9427-4e9c-b7f7-97fe10d6d52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:57.867732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b0512ac7-9427-4e9c-b7f7-97fe10d6d52a None None] MFA Rules not processed for user `aafddfa737124229b77dd87fdd3d6c22`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:57.868934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b0512ac7-9427-4e9c-b7f7-97fe10d6d52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:57.928889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 952/1882] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:06:57 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:57.934003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9e75d-6727-49de-b466-cc34194aedb7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.972767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 931/1883] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:57 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:57.973613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0512ac7-9427-4e9c-b7f7-97fe10d6d52a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:57.974790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b0512ac7-9427-4e9c-b7f7-97fe10d6d52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:58.017997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9e75d-6727-49de-b466-cc34194aedb7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:58.023307 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b7f80-8e21-4834-ab42-434dd4ed09b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:58.024032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b7f80-8e21-4834-ab42-434dd4ed09b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.024321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b7f80-8e21-4834-ab42-434dd4ed09b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.025463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b7f80-8e21-4834-ab42-434dd4ed09b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.025463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b7f80-8e21-4834-ab42-434dd4ed09b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:58.027472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b7f80-8e21-4834-ab42-434dd4ed09b9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:58.058422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0512ac7-9427-4e9c-b7f7-97fe10d6d52a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:58.063869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:58.065195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.065195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.065378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] PATH_INFO: `/v3/users/d732bc54c32d40f7ae74ca902d938f1f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.066631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d732bc54c32d40f7ae74ca902d938f1f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:58.068637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:58.146386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 953/1884] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:06:57 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:06:58.147212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b7f80-8e21-4834-ab42-434dd4ed09b9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:58.148813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-103b7f80-8e21-4834-ab42-434dd4ed09b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:58.204655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b7f80-8e21-4834-ab42-434dd4ed09b9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:58.208337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c25d2-327c-420e-8db6-3f99ee636baf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:58.208974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c25d2-327c-420e-8db6-3f99ee636baf service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.209254 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c25d2-327c-420e-8db6-3f99ee636baf service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.209527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c25d2-327c-420e-8db6-3f99ee636baf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.248790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:58.269012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c25d2-327c-420e-8db6-3f99ee636baf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:58.270466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c25d2-327c-420e-8db6-3f99ee636baf service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:58.280074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 954/1885] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:58 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:58.280692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c25d2-327c-420e-8db6-3f99ee636baf service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:58.291718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c25d2-327c-420e-8db6-3f99ee636baf service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:58.295800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd732bc54c32d40f7ae74ca902d938f1f'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:58.296330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c22f9f-f53d-4d10-8010-82522ccd29e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:58.296984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c22f9f-f53d-4d10-8010-82522ccd29e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.297262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c22f9f-f53d-4d10-8010-82522ccd29e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.298434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c22f9f-f53d-4d10-8010-82522ccd29e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.298988 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c22f9f-f53d-4d10-8010-82522ccd29e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:58.301256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c22f9f-f53d-4d10-8010-82522ccd29e1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:58.304059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd732bc54c32d40f7ae74ca902d938f1f'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:58.316250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd732bc54c32d40f7ae74ca902d938f1f'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:58.320498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 955/1886] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:06:58.324293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c22f9f-f53d-4d10-8010-82522ccd29e1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:58.335433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c22f9f-f53d-4d10-8010-82522ccd29e1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:58.340495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644473e-dd10-4781-ad11-312c0475a92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:58.341879 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644473e-dd10-4781-ad11-312c0475a92a admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.342252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644473e-dd10-4781-ad11-312c0475a92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.342585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644473e-dd10-4781-ad11-312c0475a92a admin admin] PATH_INFO: `/v3/projects/ea62842c5353476981ffd778d9f67dd7/users/d586ef5f1622460cb6e3b443274e1062/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.366755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:58.371424 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644473e-dd10-4781-ad11-312c0475a92a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea62842c5353476981ffd778d9f67dd7, user_id=d586ef5f1622460cb6e3b443274e1062, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:58.375147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644473e-dd10-4781-ad11-312c0475a92a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:58.394004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd732bc54c32d40f7ae74ca902d938f1f'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:58.395018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b9cf2e61-8601-47d3-b6de-f90cd8c36b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd732bc54c32d40f7ae74ca902d938f1f'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:58.401649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 932/1887] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:57 2022] DELETE /identity/v3/users/d732bc54c32d40f7ae74ca902d938f1f => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:58.407271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59162a-55c1-4673-8137-c31815cb87a3 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.411139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59162a-55c1-4673-8137-c31815cb87a3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.411139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59162a-55c1-4673-8137-c31815cb87a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.426956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ab59162a-55c1-4673-8137-c31815cb87a3 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:58.428224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ab59162a-55c1-4673-8137-c31815cb87a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:58.467282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 956/1888] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:58 2022] PUT /identity/v3/projects/ea62842c5353476981ffd778d9f67dd7/users/d586ef5f1622460cb6e3b443274e1062/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:58.471267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644473e-dd10-4781-ad11-312c0475a92a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:58.473451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2644473e-dd10-4781-ad11-312c0475a92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:58.507782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 933/1889] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:06:58.518464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59162a-55c1-4673-8137-c31815cb87a3 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:58.520247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ab59162a-55c1-4673-8137-c31815cb87a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:58.553448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644473e-dd10-4781-ad11-312c0475a92a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:58.558761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f52dbe-29a5-4766-ac49-8369098c1ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:58.559425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f52dbe-29a5-4766-ac49-8369098c1ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.559713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f52dbe-29a5-4766-ac49-8369098c1ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.559989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f52dbe-29a5-4766-ac49-8369098c1ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.560975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f52dbe-29a5-4766-ac49-8369098c1ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:58.562430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f52dbe-29a5-4766-ac49-8369098c1ca1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:58.573058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 957/1890] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:06:58.577249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f52dbe-29a5-4766-ac49-8369098c1ca1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:58.581112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59162a-55c1-4673-8137-c31815cb87a3 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:58.588452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f52dbe-29a5-4766-ac49-8369098c1ca1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:58.593412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4176bcd-b1ca-4e1e-bf9d-2349fd894b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:58.594299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4176bcd-b1ca-4e1e-bf9d-2349fd894b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.594593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4176bcd-b1ca-4e1e-bf9d-2349fd894b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.594911 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4176bcd-b1ca-4e1e-bf9d-2349fd894b2f admin admin] PATH_INFO: `/v3/projects/ea62842c5353476981ffd778d9f67dd7/users/d586ef5f1622460cb6e3b443274e1062/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.596973 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4176bcd-b1ca-4e1e-bf9d-2349fd894b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea62842c5353476981ffd778d9f67dd7, user_id=d586ef5f1622460cb6e3b443274e1062, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:58.599743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4176bcd-b1ca-4e1e-bf9d-2349fd894b2f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:58.610750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 958/1891] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:06:58 2022] PUT /identity/v3/projects/ea62842c5353476981ffd778d9f67dd7/users/d586ef5f1622460cb6e3b443274e1062/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:06:58.620862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd24ac-9068-4a81-b6c9-b922abf0e13e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.621457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd24ac-9068-4a81-b6c9-b922abf0e13e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.622428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd24ac-9068-4a81-b6c9-b922abf0e13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.653215 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24918f75-5c15-471d-9966-f75fdf1cac4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:58.654087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24918f75-5c15-471d-9966-f75fdf1cac4c service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.654522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24918f75-5c15-471d-9966-f75fdf1cac4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.654907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24918f75-5c15-471d-9966-f75fdf1cac4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.658571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-cebd24ac-9068-4a81-b6c9-b922abf0e13e None None] MFA Rules not processed for user `d586ef5f1622460cb6e3b443274e1062`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:06:58.660510 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cebd24ac-9068-4a81-b6c9-b922abf0e13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:58.666468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24918f75-5c15-471d-9966-f75fdf1cac4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:58.668108 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24918f75-5c15-471d-9966-f75fdf1cac4c service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:58.679713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 934/1892] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:06:58.744896 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 959/1893] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:06:58 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:06:58.760283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24918f75-5c15-471d-9966-f75fdf1cac4c service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:58.761813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-24918f75-5c15-471d-9966-f75fdf1cac4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:58.784990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd24ac-9068-4a81-b6c9-b922abf0e13e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:58.787028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cebd24ac-9068-4a81-b6c9-b922abf0e13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:58.843895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24918f75-5c15-471d-9966-f75fdf1cac4c service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:58.889458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd24ac-9068-4a81-b6c9-b922abf0e13e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:58.894336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e8462-c3c3-4ac1-98f5-7b353157518c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:58.895260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e8462-c3c3-4ac1-98f5-7b353157518c service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.895681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e8462-c3c3-4ac1-98f5-7b353157518c service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.896121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e8462-c3c3-4ac1-98f5-7b353157518c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.898142 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3c7e8462-c3c3-4ac1-98f5-7b353157518c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:58.925557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b817001-4f24-483b-ad9b-13165c6a3d99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:58.925557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b817001-4f24-483b-ad9b-13165c6a3d99 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:58.925897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b817001-4f24-483b-ad9b-13165c6a3d99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:58.925982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b817001-4f24-483b-ad9b-13165c6a3d99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:58.938321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b817001-4f24-483b-ad9b-13165c6a3d99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:58.940370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b817001-4f24-483b-ad9b-13165c6a3d99 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:58.959813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 935/1894] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:58 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:58.972542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e8462-c3c3-4ac1-98f5-7b353157518c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:58.974057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e8462-c3c3-4ac1-98f5-7b353157518c service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:58.986286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 960/1895] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:58 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:06:59.027607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:59.256168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:59.295648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b817001-4f24-483b-ad9b-13165c6a3d99 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:59.297171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9b817001-4f24-483b-ad9b-13165c6a3d99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:59.383687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b817001-4f24-483b-ad9b-13165c6a3d99 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:59.390226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:59.396396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:59.396396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:59.396396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] PATH_INFO: `/v3/projects/c836a23bb10c4a9aaa1e615076c184e6` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:59.396396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c836a23bb10c4a9aaa1e615076c184e6)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:59.396396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:59.492913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e8462-c3c3-4ac1-98f5-7b353157518c service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:59.502361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e8462-c3c3-4ac1-98f5-7b353157518c service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:59.507716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cda18-7955-489f-a531-6df3aab4b73a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:59.508462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cda18-7955-489f-a531-6df3aab4b73a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:59.508823 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cda18-7955-489f-a531-6df3aab4b73a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:59.509186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cda18-7955-489f-a531-6df3aab4b73a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:59.511292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-901cda18-7955-489f-a531-6df3aab4b73a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:06:59.546977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c836a23bb10c4a9aaa1e615076c184e6', 'request_id': 'req-19c57fee-b492-4837-971d-87ee7d132cc8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:59.557575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c836a23bb10c4a9aaa1e615076c184e6', 'request_id': 'req-19c57fee-b492-4837-971d-87ee7d132cc8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:59.572019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:59.599483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901cda18-7955-489f-a531-6df3aab4b73a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:59.600983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901cda18-7955-489f-a531-6df3aab4b73a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:59.611485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 961/1896] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:06:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:06:59.657358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c836a23bb10c4a9aaa1e615076c184e6', 'request_id': 'req-19c57fee-b492-4837-971d-87ee7d132cc8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:59.671853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] The token cache is being invalidate because project c836a23bb10c4a9aaa1e615076c184e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:06:59.672925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:59.674352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:59.676223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 936/1897] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:06:59 2022] DELETE /identity/v3/projects/c836a23bb10c4a9aaa1e615076c184e6 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:06:59.680571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cda18-7955-489f-a531-6df3aab4b73a service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:06:59.691898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cda18-7955-489f-a531-6df3aab4b73a service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:06:59.747618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:06:59.751634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:06:59.751634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:06:59.751634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] PATH_INFO: `/v3/users/1f4538e72fd44b87bf2dc220cd906783` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:06:59.751634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f4538e72fd44b87bf2dc220cd906783)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:06:59.752668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:06:59.889080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:06:59.964353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f4538e72fd44b87bf2dc220cd906783'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:06:59.972683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f4538e72fd44b87bf2dc220cd906783'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:00.118508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f4538e72fd44b87bf2dc220cd906783'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:00.119243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f4538e72fd44b87bf2dc220cd906783'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:00.119957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e17da81d-57de-4fd0-bbd2-8af5c12e8423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f4538e72fd44b87bf2dc220cd906783'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:00.148875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 962/1898] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:06:59 2022] DELETE /identity/v3/users/1f4538e72fd44b87bf2dc220cd906783 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:00.508194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:01.083306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:01.084813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:01.105268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c57fee-b492-4837-971d-87ee7d132cc8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:01.164063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:01.168553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:01.168553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:01.168553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] PATH_INFO: `/v3/projects/6163149bce154604b6a9a4ff3ed95c8f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:01.168553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6163149bce154604b6a9a4ff3ed95c8f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:01.168553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:01.242161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:01.295679 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6163149bce154604b6a9a4ff3ed95c8f', 'request_id': 'req-623e1574-9864-4d99-97a9-25b909b47cbf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:01.303408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6163149bce154604b6a9a4ff3ed95c8f', 'request_id': 'req-623e1574-9864-4d99-97a9-25b909b47cbf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:01.393316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6163149bce154604b6a9a4ff3ed95c8f', 'request_id': 'req-623e1574-9864-4d99-97a9-25b909b47cbf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:01.412116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] The token cache is being invalidate because project 6163149bce154604b6a9a4ff3ed95c8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:01.413868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:01.416050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-623e1574-9864-4d99-97a9-25b909b47cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:01.425677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 937/1899] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:01 2022] DELETE /identity/v3/projects/6163149bce154604b6a9a4ff3ed95c8f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:01.427436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b5580-b373-4b9f-910b-b6d957978457 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:01.427750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b5580-b373-4b9f-910b-b6d957978457 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:01.428027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b5580-b373-4b9f-910b-b6d957978457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:01.450862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-be4b5580-b373-4b9f-910b-b6d957978457 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:01.452074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-be4b5580-b373-4b9f-910b-b6d957978457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:01.519829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 963/1900] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:01.527522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918fb40-b7b6-43db-8438-04ab8e62995c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:01.528730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918fb40-b7b6-43db-8438-04ab8e62995c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:01.529096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918fb40-b7b6-43db-8438-04ab8e62995c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:01.551185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7918fb40-b7b6-43db-8438-04ab8e62995c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:01.552435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7918fb40-b7b6-43db-8438-04ab8e62995c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:01.559443 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:01.562827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 938/1901] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:01.566123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b5580-b373-4b9f-910b-b6d957978457 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:01.576031 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b5580-b373-4b9f-910b-b6d957978457 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:01.581471 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a30f63-06e4-41cc-a04d-74e2518c798e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:01.585815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a30f63-06e4-41cc-a04d-74e2518c798e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:01.585815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a30f63-06e4-41cc-a04d-74e2518c798e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:01.585815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a30f63-06e4-41cc-a04d-74e2518c798e admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:01.585815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a30f63-06e4-41cc-a04d-74e2518c798e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:01.585815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a30f63-06e4-41cc-a04d-74e2518c798e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:01.585815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a30f63-06e4-41cc-a04d-74e2518c798e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:01.600364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 964/1902] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:07:01.604945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918fb40-b7b6-43db-8438-04ab8e62995c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:01.614787 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918fb40-b7b6-43db-8438-04ab8e62995c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:01.620184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134818-fd7f-4fc6-b3f1-b0dc87511c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:01.620674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134818-fd7f-4fc6-b3f1-b0dc87511c2c admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:01.621102 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134818-fd7f-4fc6-b3f1-b0dc87511c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:01.621390 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134818-fd7f-4fc6-b3f1-b0dc87511c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:01.622166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f134818-fd7f-4fc6-b3f1-b0dc87511c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:01.624091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f134818-fd7f-4fc6-b3f1-b0dc87511c2c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:01.773970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 939/1903] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:01 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:01.778552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a30f63-06e4-41cc-a04d-74e2518c798e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:01.800370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a30f63-06e4-41cc-a04d-74e2518c798e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:01.864311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397488c-ad6e-4a1e-8d1c-132957be901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:01.865098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397488c-ad6e-4a1e-8d1c-132957be901a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:01.865432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397488c-ad6e-4a1e-8d1c-132957be901a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:01.865811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397488c-ad6e-4a1e-8d1c-132957be901a admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:01.866522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397488c-ad6e-4a1e-8d1c-132957be901a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:01.868459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397488c-ad6e-4a1e-8d1c-132957be901a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:01.983928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 965/1904] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:01 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:01.988461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134818-fd7f-4fc6-b3f1-b0dc87511c2c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:02.001034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134818-fd7f-4fc6-b3f1-b0dc87511c2c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:02.006017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a504dca-62b3-48da-99ba-0dd7cae15207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:02.006742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a504dca-62b3-48da-99ba-0dd7cae15207 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.007027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a504dca-62b3-48da-99ba-0dd7cae15207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.007294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a504dca-62b3-48da-99ba-0dd7cae15207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:02.008027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a504dca-62b3-48da-99ba-0dd7cae15207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:02.009666 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a504dca-62b3-48da-99ba-0dd7cae15207 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:02.021714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 940/1905] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:02.027675 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397488c-ad6e-4a1e-8d1c-132957be901a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:02.036834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397488c-ad6e-4a1e-8d1c-132957be901a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:02.039666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:02.043222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dd608-dd5e-4d79-ae30-84fe413a9e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:02.043947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd608-dd5e-4d79-ae30-84fe413a9e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.044200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd608-dd5e-4d79-ae30-84fe413a9e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.044461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dd608-dd5e-4d79-ae30-84fe413a9e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:02.045191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dd608-dd5e-4d79-ae30-84fe413a9e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:02.049349 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dd608-dd5e-4d79-ae30-84fe413a9e42 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:02.058330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 966/1906] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:02.063221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a504dca-62b3-48da-99ba-0dd7cae15207 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:02.073348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a504dca-62b3-48da-99ba-0dd7cae15207 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:02.077933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03621040-8296-4649-bfee-df8f9141f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:02.079490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03621040-8296-4649-bfee-df8f9141f332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.079778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03621040-8296-4649-bfee-df8f9141f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.080054 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03621040-8296-4649-bfee-df8f9141f332 admin admin] PATH_INFO: `/v3/projects/a02f5833f666469d920f62cf415bf827/users/762233386e3047e5892e608ffec0d955/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:02.096150 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03621040-8296-4649-bfee-df8f9141f332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02f5833f666469d920f62cf415bf827, user_id=762233386e3047e5892e608ffec0d955, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:02.098942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03621040-8296-4649-bfee-df8f9141f332 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:02.192271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 941/1907] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:02 2022] PUT /identity/v3/projects/a02f5833f666469d920f62cf415bf827/users/762233386e3047e5892e608ffec0d955/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:02.202739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe8dff-1e7c-41a1-acea-5b393475f326 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.203036 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe8dff-1e7c-41a1-acea-5b393475f326 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.203336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe8dff-1e7c-41a1-acea-5b393475f326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:02.230185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bdfe8dff-1e7c-41a1-acea-5b393475f326 None None] MFA Rules not processed for user `762233386e3047e5892e608ffec0d955`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:02.231411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bdfe8dff-1e7c-41a1-acea-5b393475f326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:02.296000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 967/1908] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:02.301235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03621040-8296-4649-bfee-df8f9141f332 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:02.331775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:02.371385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03621040-8296-4649-bfee-df8f9141f332 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:02.376884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e8398-0650-488d-951c-425d22c9c922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:02.377538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e8398-0650-488d-951c-425d22c9c922 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.377833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e8398-0650-488d-951c-425d22c9c922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.378112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e8398-0650-488d-951c-425d22c9c922 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:02.378799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e8398-0650-488d-951c-425d22c9c922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:02.380683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e8398-0650-488d-951c-425d22c9c922 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:02.477339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 942/1909] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:02 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:02.478527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe8dff-1e7c-41a1-acea-5b393475f326 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:02.490460 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe8dff-1e7c-41a1-acea-5b393475f326 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:02.551214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8a7b6-e2e9-45af-877a-9603a61bcccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:02.551879 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8a7b6-e2e9-45af-877a-9603a61bcccc admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.552168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8a7b6-e2e9-45af-877a-9603a61bcccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.552442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8a7b6-e2e9-45af-877a-9603a61bcccc admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:02.553425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8a7b6-e2e9-45af-877a-9603a61bcccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:02.555425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8a7b6-e2e9-45af-877a-9603a61bcccc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:02.587234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:02.666117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 968/1910] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:02 2022] POST /identity/v3/users => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:02.668342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e8398-0650-488d-951c-425d22c9c922 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:02.681088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e8398-0650-488d-951c-425d22c9c922 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:02.685835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ef8e2-868a-421e-a4b8-b9573fd55468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:02.686541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ef8e2-868a-421e-a4b8-b9573fd55468 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.686829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ef8e2-868a-421e-a4b8-b9573fd55468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.687107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ef8e2-868a-421e-a4b8-b9573fd55468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:02.687894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ef8e2-868a-421e-a4b8-b9573fd55468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:02.689562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ef8e2-868a-421e-a4b8-b9573fd55468 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:02.700768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 943/1911] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:02.704723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8a7b6-e2e9-45af-877a-9603a61bcccc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:02.715541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8a7b6-e2e9-45af-877a-9603a61bcccc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:02.721027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac5867-e9a5-49f4-8411-d785cfa7afe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:02.721925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac5867-e9a5-49f4-8411-d785cfa7afe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.722245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac5867-e9a5-49f4-8411-d785cfa7afe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.722584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac5867-e9a5-49f4-8411-d785cfa7afe5 admin admin] PATH_INFO: `/v3/projects/0b7f7ef1499c42b387b27b91a916aba0/users/22051f5aa970411da254e4a05e7d73e2/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:02.742100 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac5867-e9a5-49f4-8411-d785cfa7afe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7f7ef1499c42b387b27b91a916aba0, user_id=22051f5aa970411da254e4a05e7d73e2, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:02.744747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac5867-e9a5-49f4-8411-d785cfa7afe5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:02.747397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:02.826545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 969/1912] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:02 2022] PUT /identity/v3/projects/0b7f7ef1499c42b387b27b91a916aba0/users/22051f5aa970411da254e4a05e7d73e2/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:02.827371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ef8e2-868a-421e-a4b8-b9573fd55468 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:02.838359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ef8e2-868a-421e-a4b8-b9573fd55468 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:02.912788 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064a575-87b0-43af-8495-52fd4dbafaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:02.913445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064a575-87b0-43af-8495-52fd4dbafaeb admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.913724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064a575-87b0-43af-8495-52fd4dbafaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.913996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064a575-87b0-43af-8495-52fd4dbafaeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:02.914764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064a575-87b0-43af-8495-52fd4dbafaeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:02.917115 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064a575-87b0-43af-8495-52fd4dbafaeb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:02.927432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 944/1913] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:02.931419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac5867-e9a5-49f4-8411-d785cfa7afe5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:02.944760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac5867-e9a5-49f4-8411-d785cfa7afe5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:02.950067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6846843-cce0-4d38-abc1-4877b677357b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:02.950965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6846843-cce0-4d38-abc1-4877b677357b admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.951246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6846843-cce0-4d38-abc1-4877b677357b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.951525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6846843-cce0-4d38-abc1-4877b677357b admin admin] PATH_INFO: `/v3/projects/0b7f7ef1499c42b387b27b91a916aba0/users/22051f5aa970411da254e4a05e7d73e2/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:02.954308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6846843-cce0-4d38-abc1-4877b677357b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7f7ef1499c42b387b27b91a916aba0, user_id=22051f5aa970411da254e4a05e7d73e2, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:02.957419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6846843-cce0-4d38-abc1-4877b677357b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:02.970561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 970/1914] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:02 2022] PUT /identity/v3/projects/0b7f7ef1499c42b387b27b91a916aba0/users/22051f5aa970411da254e4a05e7d73e2/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:02.981859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73816701-a617-421a-a194-b30da71f06e8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:02.982183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73816701-a617-421a-a194-b30da71f06e8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:02.982803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73816701-a617-421a-a194-b30da71f06e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:03.012614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-73816701-a617-421a-a194-b30da71f06e8 None None] MFA Rules not processed for user `22051f5aa970411da254e4a05e7d73e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:03.013843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-73816701-a617-421a-a194-b30da71f06e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:03.088261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 945/1915] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:03.100390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6846843-cce0-4d38-abc1-4877b677357b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:03.102086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d6846843-cce0-4d38-abc1-4877b677357b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:03.161745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6846843-cce0-4d38-abc1-4877b677357b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:03.165371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da694e-f459-4c93-abe8-76c3e9a364a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:03.166137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da694e-f459-4c93-abe8-76c3e9a364a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:03.166451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da694e-f459-4c93-abe8-76c3e9a364a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:03.166728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da694e-f459-4c93-abe8-76c3e9a364a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:03.176504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da694e-f459-4c93-abe8-76c3e9a364a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:03.177931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da694e-f459-4c93-abe8-76c3e9a364a4 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:03.187704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 971/1916] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:03.360337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73816701-a617-421a-a194-b30da71f06e8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:03.369947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73816701-a617-421a-a194-b30da71f06e8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:03.373806 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0cb578-74f4-4ed7-973c-5744b35bf6bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:03.374498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cb578-74f4-4ed7-973c-5744b35bf6bc service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:03.382579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cb578-74f4-4ed7-973c-5744b35bf6bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:03.382579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cb578-74f4-4ed7-973c-5744b35bf6bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:03.417178 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:03.432808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0cb578-74f4-4ed7-973c-5744b35bf6bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:03.434259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0cb578-74f4-4ed7-973c-5744b35bf6bc service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:03.443507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 946/1917] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:03.988459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da694e-f459-4c93-abe8-76c3e9a364a4 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:04.057753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da694e-f459-4c93-abe8-76c3e9a364a4 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:04.062374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:04.063895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:04.064174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:04.064444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] PATH_INFO: `/v3/users/762233386e3047e5892e608ffec0d955` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:04.065667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=762233386e3047e5892e608ffec0d955)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:04.067661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:04.298862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762233386e3047e5892e608ffec0d955'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:04.305615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762233386e3047e5892e608ffec0d955'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:04.460127 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762233386e3047e5892e608ffec0d955'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:04.460127 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762233386e3047e5892e608ffec0d955'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:04.460127 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762233386e3047e5892e608ffec0d955'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:04.483838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 972/1918] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:03 2022] DELETE /identity/v3/users/762233386e3047e5892e608ffec0d955 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:04.489303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a42fc7-8bfc-4e90-9f3f-a0b41b9e5fbd None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:04.489817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a42fc7-8bfc-4e90-9f3f-a0b41b9e5fbd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:04.489914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a42fc7-8bfc-4e90-9f3f-a0b41b9e5fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:04.507791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-79a42fc7-8bfc-4e90-9f3f-a0b41b9e5fbd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:04.508980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-79a42fc7-8bfc-4e90-9f3f-a0b41b9e5fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:04.571549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 947/1919] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:04.581765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:04.583188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:04.588508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:04.642085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388be7c-9a74-461b-80de-b13fe02fd6eb admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:04.645963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1b85f-c0b3-4e5e-90cb-d427209271f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:04.646641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1b85f-c0b3-4e5e-90cb-d427209271f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:04.646924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1b85f-c0b3-4e5e-90cb-d427209271f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:04.647242 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1b85f-c0b3-4e5e-90cb-d427209271f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:04.656276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1b85f-c0b3-4e5e-90cb-d427209271f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:04.658661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1b85f-c0b3-4e5e-90cb-d427209271f2 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:04.667898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 973/1920] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:05.056323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a42fc7-8bfc-4e90-9f3f-a0b41b9e5fbd None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:05.057735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-79a42fc7-8bfc-4e90-9f3f-a0b41b9e5fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:05.071442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a42fc7-8bfc-4e90-9f3f-a0b41b9e5fbd None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:05.076355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:05.077234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:05.077722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:05.078147 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] PATH_INFO: `/v3/users/2fdaf317517b4b3cb097be5aa00ab5c0` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:05.081954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fdaf317517b4b3cb097be5aa00ab5c0)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:05.082932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:05.112462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1b85f-c0b3-4e5e-90cb-d427209271f2 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:05.114158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5ee1b85f-c0b3-4e5e-90cb-d427209271f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:05.135918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1b85f-c0b3-4e5e-90cb-d427209271f2 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:05.150178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:05.151155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:05.151504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:05.151848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] PATH_INFO: `/v3/projects/a02f5833f666469d920f62cf415bf827` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:05.153140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a02f5833f666469d920f62cf415bf827)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:05.155280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:05.322206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fdaf317517b4b3cb097be5aa00ab5c0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.322896 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a02f5833f666469d920f62cf415bf827', 'request_id': 'req-57ccefb8-d053-434b-9705-17dbb544260b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.328608 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a02f5833f666469d920f62cf415bf827', 'request_id': 'req-57ccefb8-d053-434b-9705-17dbb544260b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.330180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fdaf317517b4b3cb097be5aa00ab5c0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.337998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a02f5833f666469d920f62cf415bf827', 'request_id': 'req-57ccefb8-d053-434b-9705-17dbb544260b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.347691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fdaf317517b4b3cb097be5aa00ab5c0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.409359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] The token cache is being invalidate because project a02f5833f666469d920f62cf415bf827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:05.409646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.410371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.411914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 974/1921] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:05 2022] DELETE /identity/v3/projects/a02f5833f666469d920f62cf415bf827 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:05.415797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:05.417058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:05.461181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fdaf317517b4b3cb097be5aa00ab5c0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.461956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-18a71d58-7bdc-4abf-a860-bcb537e762cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fdaf317517b4b3cb097be5aa00ab5c0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.469732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 948/1922] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:05 2022] DELETE /identity/v3/users/2fdaf317517b4b3cb097be5aa00ab5c0 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:05.474117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1c77e-d6be-4f56-96f8-e54978d8f606 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:05.476286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1c77e-d6be-4f56-96f8-e54978d8f606 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:05.476286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1c77e-d6be-4f56-96f8-e54978d8f606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:05.483149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccefb8-d053-434b-9705-17dbb544260b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:05.491949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-30e1c77e-d6be-4f56-96f8-e54978d8f606 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:05.493120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-30e1c77e-d6be-4f56-96f8-e54978d8f606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:05.517245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:05.541941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:05.542169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:05.543134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:05.543414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:05.543688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] PATH_INFO: `/v3/users/22051f5aa970411da254e4a05e7d73e2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:05.544922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22051f5aa970411da254e4a05e7d73e2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:05.546907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:05.554110 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 949/1923] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:05.563591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1c77e-d6be-4f56-96f8-e54978d8f606 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:05.565025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-30e1c77e-d6be-4f56-96f8-e54978d8f606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:05.622345 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1c77e-d6be-4f56-96f8-e54978d8f606 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:05.627193 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f7307-7f79-423f-9b00-7bd9616b4b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:05.627846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f7307-7f79-423f-9b00-7bd9616b4b1d service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:05.628133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f7307-7f79-423f-9b00-7bd9616b4b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:05.628409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f7307-7f79-423f-9b00-7bd9616b4b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:05.637588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f7307-7f79-423f-9b00-7bd9616b4b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:05.639039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f7307-7f79-423f-9b00-7bd9616b4b1d service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:05.649100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 950/1924] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:05.760349 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22051f5aa970411da254e4a05e7d73e2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.767869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22051f5aa970411da254e4a05e7d73e2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.833028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22051f5aa970411da254e4a05e7d73e2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.833907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22051f5aa970411da254e4a05e7d73e2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.834621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22051f5aa970411da254e4a05e7d73e2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:05.853316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 975/1925] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:05 2022] DELETE /identity/v3/users/22051f5aa970411da254e4a05e7d73e2 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:06.097732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f7307-7f79-423f-9b00-7bd9616b4b1d service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:06.099985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-824f7307-7f79-423f-9b00-7bd9616b4b1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:06.116382 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f7307-7f79-423f-9b00-7bd9616b4b1d service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:06.181912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:06.182825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:06.183106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:06.183381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] PATH_INFO: `/v3/projects/746c0db8002347f48bd3c203c6616ed8` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:06.184566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=746c0db8002347f48bd3c203c6616ed8)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:06.186514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:06.269711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '746c0db8002347f48bd3c203c6616ed8', 'request_id': 'req-f36cd902-3078-44cb-9551-637bd505f083'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.275728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '746c0db8002347f48bd3c203c6616ed8', 'request_id': 'req-f36cd902-3078-44cb-9551-637bd505f083'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.288300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:06.288771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:06.308441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc6b1-c109-45e1-be50-f7fa46139946 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:06.364027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '746c0db8002347f48bd3c203c6616ed8', 'request_id': 'req-f36cd902-3078-44cb-9551-637bd505f083'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.377700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:06.378205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] The token cache is being invalidate because project 746c0db8002347f48bd3c203c6616ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:06.378500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.378622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:06.378882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:06.379148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.379357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] PATH_INFO: `/v3/projects/0b7f7ef1499c42b387b27b91a916aba0` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:06.380365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7f7ef1499c42b387b27b91a916aba0)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:06.381600 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 951/1926] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:06 2022] DELETE /identity/v3/projects/746c0db8002347f48bd3c203c6616ed8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:06.382457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:06.386148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:06.396196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:06.416984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cd902-3078-44cb-9551-637bd505f083 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:06.422687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:06.428331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:06.428331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:06.428331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] PATH_INFO: `/v3/users/d586ef5f1622460cb6e3b443274e1062` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:06.428331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d586ef5f1622460cb6e3b443274e1062)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:06.428331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:06.450342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:06.495932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7f7ef1499c42b387b27b91a916aba0', 'request_id': 'req-1af1a418-3f17-4b43-bf74-e45002b2f240'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.500783 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:06.501966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7f7ef1499c42b387b27b91a916aba0', 'request_id': 'req-1af1a418-3f17-4b43-bf74-e45002b2f240'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.508474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7f7ef1499c42b387b27b91a916aba0', 'request_id': 'req-1af1a418-3f17-4b43-bf74-e45002b2f240'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.569564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] The token cache is being invalidate because project 0b7f7ef1499c42b387b27b91a916aba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:06.571170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.571170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1af1a418-3f17-4b43-bf74-e45002b2f240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.572160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 976/1927] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:06 2022] DELETE /identity/v3/projects/0b7f7ef1499c42b387b27b91a916aba0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:06.576706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e3a95-63c9-4a37-a230-b19470ac6b17 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:06.576998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e3a95-63c9-4a37-a230-b19470ac6b17 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:06.577275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e3a95-63c9-4a37-a230-b19470ac6b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:06.596889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3f7e3a95-63c9-4a37-a230-b19470ac6b17 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:06.599852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3f7e3a95-63c9-4a37-a230-b19470ac6b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:06.633398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd586ef5f1622460cb6e3b443274e1062'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.640066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd586ef5f1622460cb6e3b443274e1062'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.653166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd586ef5f1622460cb6e3b443274e1062'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.658929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 977/1928] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:06.666495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fc307-e8b2-4044-9e2f-bd0d52ed2b9e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:06.666808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fc307-e8b2-4044-9e2f-bd0d52ed2b9e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:06.667086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fc307-e8b2-4044-9e2f-bd0d52ed2b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:06.685213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d92fc307-e8b2-4044-9e2f-bd0d52ed2b9e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:06.686426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d92fc307-e8b2-4044-9e2f-bd0d52ed2b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:06.694593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 978/1929] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:06.698500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fc307-e8b2-4044-9e2f-bd0d52ed2b9e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:06.707160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fc307-e8b2-4044-9e2f-bd0d52ed2b9e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:06.711664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2338f7-f5ab-4719-927d-ad336eeaf8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:06.712276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2338f7-f5ab-4719-927d-ad336eeaf8be admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:06.712556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2338f7-f5ab-4719-927d-ad336eeaf8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:06.712830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2338f7-f5ab-4719-927d-ad336eeaf8be admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:06.713528 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2338f7-f5ab-4719-927d-ad336eeaf8be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:06.713831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2338f7-f5ab-4719-927d-ad336eeaf8be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:06.715434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2338f7-f5ab-4719-927d-ad336eeaf8be admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:06.723655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd586ef5f1622460cb6e3b443274e1062'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.724267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd586ef5f1622460cb6e3b443274e1062'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:06.727536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 979/1930] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:07:06.730848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 952/1931] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:06 2022] DELETE /identity/v3/users/d586ef5f1622460cb6e3b443274e1062 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:06.731640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2338f7-f5ab-4719-927d-ad336eeaf8be admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:06.733223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ec2338f7-f5ab-4719-927d-ad336eeaf8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:06.747196 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2338f7-f5ab-4719-927d-ad336eeaf8be admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:06.805508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a19ce4-6563-4065-80ed-60b50e9ab185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:06.806232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a19ce4-6563-4065-80ed-60b50e9ab185 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:06.806519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a19ce4-6563-4065-80ed-60b50e9ab185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:06.806797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a19ce4-6563-4065-80ed-60b50e9ab185 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:06.807568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a19ce4-6563-4065-80ed-60b50e9ab185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:06.809571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a19ce4-6563-4065-80ed-60b50e9ab185 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:06.907549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 980/1932] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:06 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:06.911802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:06.922463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72422061-cd5e-4859-904e-6b476e137655 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:06.992824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a5dac-d2fc-4641-9246-81c2a0d34bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:06.993503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a5dac-d2fc-4641-9246-81c2a0d34bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:06.993810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a5dac-d2fc-4641-9246-81c2a0d34bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:06.994114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a5dac-d2fc-4641-9246-81c2a0d34bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:06.994804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a5dac-d2fc-4641-9246-81c2a0d34bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:06.996725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a5dac-d2fc-4641-9246-81c2a0d34bb0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.112585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 953/1933] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:06 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:07.117339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a19ce4-6563-4065-80ed-60b50e9ab185 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.128190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a19ce4-6563-4065-80ed-60b50e9ab185 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.132710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94713c0e-b50e-4462-98ee-f1a795033e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.133367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713c0e-b50e-4462-98ee-f1a795033e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.138838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713c0e-b50e-4462-98ee-f1a795033e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.138838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713c0e-b50e-4462-98ee-f1a795033e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.138838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94713c0e-b50e-4462-98ee-f1a795033e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.138838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94713c0e-b50e-4462-98ee-f1a795033e67 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.143269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a5dac-d2fc-4641-9246-81c2a0d34bb0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.145318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-035a5dac-d2fc-4641-9246-81c2a0d34bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:07.151636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 981/1934] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:07.155601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94713c0e-b50e-4462-98ee-f1a795033e67 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.164825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a5dac-d2fc-4641-9246-81c2a0d34bb0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.167398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94713c0e-b50e-4462-98ee-f1a795033e67 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.173919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.173919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.173919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.173919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] PATH_INFO: `/v3/projects/ea62842c5353476981ffd778d9f67dd7` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.174572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242e860-0308-4c6f-b9b8-864d44a0df9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.174572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242e860-0308-4c6f-b9b8-864d44a0df9a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.174809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea62842c5353476981ffd778d9f67dd7)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.174882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242e860-0308-4c6f-b9b8-864d44a0df9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.174882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242e860-0308-4c6f-b9b8-864d44a0df9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.175399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242e860-0308-4c6f-b9b8-864d44a0df9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.176712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.177444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242e860-0308-4c6f-b9b8-864d44a0df9a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.189149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 982/1935] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:07.193020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242e860-0308-4c6f-b9b8-864d44a0df9a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.202829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242e860-0308-4c6f-b9b8-864d44a0df9a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.208294 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cff893-6e7b-4f27-ac2d-1a3ab311adee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.209177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cff893-6e7b-4f27-ac2d-1a3ab311adee admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.209467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cff893-6e7b-4f27-ac2d-1a3ab311adee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.209743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cff893-6e7b-4f27-ac2d-1a3ab311adee admin admin] PATH_INFO: `/v3/projects/903774042ada4832a68353809fe4b5d8/users/51aef575529144b584e5dbf487641e0e/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.227811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cff893-6e7b-4f27-ac2d-1a3ab311adee admin admin] RBAC: Authorizing `identity:create_grant(project_id=903774042ada4832a68353809fe4b5d8, user_id=51aef575529144b584e5dbf487641e0e, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.230515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cff893-6e7b-4f27-ac2d-1a3ab311adee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.349904 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 983/1936] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:07 2022] PUT /identity/v3/projects/903774042ada4832a68353809fe4b5d8/users/51aef575529144b584e5dbf487641e0e/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:07.357387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea62842c5353476981ffd778d9f67dd7', 'request_id': 'req-42f91fbe-06b4-407f-8fc9-f75cc874886e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:07.360889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dc46e-e9b6-4a49-8f43-ac79b7496637 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.361193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dc46e-e9b6-4a49-8f43-ac79b7496637 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.361466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dc46e-e9b6-4a49-8f43-ac79b7496637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.364260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea62842c5353476981ffd778d9f67dd7', 'request_id': 'req-42f91fbe-06b4-407f-8fc9-f75cc874886e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:07.368207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:07.387412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-882dc46e-e9b6-4a49-8f43-ac79b7496637 None None] MFA Rules not processed for user `51aef575529144b584e5dbf487641e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:07.388542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-882dc46e-e9b6-4a49-8f43-ac79b7496637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:07.455386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 984/1937] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:07.459945 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dc46e-e9b6-4a49-8f43-ac79b7496637 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.495815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea62842c5353476981ffd778d9f67dd7', 'request_id': 'req-42f91fbe-06b4-407f-8fc9-f75cc874886e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:07.510175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] The token cache is being invalidate because project ea62842c5353476981ffd778d9f67dd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:07.510468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:07.511203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42f91fbe-06b4-407f-8fc9-f75cc874886e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:07.517400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 954/1938] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:07 2022] DELETE /identity/v3/projects/ea62842c5353476981ffd778d9f67dd7 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:07.518823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dc2dd-448f-42ce-be00-4e1fca9623b9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.519113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dc2dd-448f-42ce-be00-4e1fca9623b9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.519471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dc2dd-448f-42ce-be00-4e1fca9623b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.528173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dc46e-e9b6-4a49-8f43-ac79b7496637 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.533840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e46022-9b23-4738-921c-0a3b453f56d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.534554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e46022-9b23-4738-921c-0a3b453f56d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.534842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e46022-9b23-4738-921c-0a3b453f56d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.535122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e46022-9b23-4738-921c-0a3b453f56d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.535827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e46022-9b23-4738-921c-0a3b453f56d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.537858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e46022-9b23-4738-921c-0a3b453f56d7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.539991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b14dc2dd-448f-42ce-be00-4e1fca9623b9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:07.541274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b14dc2dd-448f-42ce-be00-4e1fca9623b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:07.549948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 955/1939] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:07.557555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dfde8-84cd-403b-8d6a-116b2a0efc70 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.560937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dfde8-84cd-403b-8d6a-116b2a0efc70 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.560937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dfde8-84cd-403b-8d6a-116b2a0efc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.576281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=14,keystone:SELECT=81,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:07.589175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-575dfde8-84cd-403b-8d6a-116b2a0efc70 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:07.590421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-575dfde8-84cd-403b-8d6a-116b2a0efc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:07.599409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 956/1940] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:07.603365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dfde8-84cd-403b-8d6a-116b2a0efc70 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.612885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dfde8-84cd-403b-8d6a-116b2a0efc70 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.618086 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8ca1d-cb16-4177-aa51-f656bc3c6aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.619031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8ca1d-cb16-4177-aa51-f656bc3c6aec admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.619324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8ca1d-cb16-4177-aa51-f656bc3c6aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.619601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8ca1d-cb16-4177-aa51-f656bc3c6aec admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.620335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8ca1d-cb16-4177-aa51-f656bc3c6aec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:07.620772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8ca1d-cb16-4177-aa51-f656bc3c6aec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.622557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8ca1d-cb16-4177-aa51-f656bc3c6aec admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.638772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 957/1941] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:07:07.639713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8ca1d-cb16-4177-aa51-f656bc3c6aec admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.652313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8ca1d-cb16-4177-aa51-f656bc3c6aec admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.658740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c341b1-d9d3-4e0a-88cb-98724398ee14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.659457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c341b1-d9d3-4e0a-88cb-98724398ee14 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.659801 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c341b1-d9d3-4e0a-88cb-98724398ee14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.660143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c341b1-d9d3-4e0a-88cb-98724398ee14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.660843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c341b1-d9d3-4e0a-88cb-98724398ee14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.662952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c341b1-d9d3-4e0a-88cb-98724398ee14 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.663137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 985/1942] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:07 2022] POST /identity/v3/projects => generated 377 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:07.667416 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e46022-9b23-4738-921c-0a3b453f56d7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.669538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-26e46022-9b23-4738-921c-0a3b453f56d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:07.756997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e46022-9b23-4738-921c-0a3b453f56d7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.761975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bdba6-9b05-4d64-8767-cce0a2445cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.762371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bdba6-9b05-4d64-8767-cce0a2445cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.762659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bdba6-9b05-4d64-8767-cce0a2445cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.762938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bdba6-9b05-4d64-8767-cce0a2445cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.763631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bdba6-9b05-4d64-8767-cce0a2445cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.766466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bdba6-9b05-4d64-8767-cce0a2445cd6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.787349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 958/1943] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:07 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:07.791097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c341b1-d9d3-4e0a-88cb-98724398ee14 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.855382 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c341b1-d9d3-4e0a-88cb-98724398ee14 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.856931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 986/1944] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:07 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:07.861598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bdba6-9b05-4d64-8767-cce0a2445cd6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.861735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8577dca-6cd7-439c-b4f9-7159ceefe7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.862367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8577dca-6cd7-439c-b4f9-7159ceefe7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.862649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8577dca-6cd7-439c-b4f9-7159ceefe7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.862926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8577dca-6cd7-439c-b4f9-7159ceefe7d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.863618 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8577dca-6cd7-439c-b4f9-7159ceefe7d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.865622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8577dca-6cd7-439c-b4f9-7159ceefe7d7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.871802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bdba6-9b05-4d64-8767-cce0a2445cd6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.877320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c357e8-fd8c-4eb2-b346-b6fc1723bd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.878066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c357e8-fd8c-4eb2-b346-b6fc1723bd9d admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.878350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c357e8-fd8c-4eb2-b346-b6fc1723bd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.878629 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c357e8-fd8c-4eb2-b346-b6fc1723bd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.879355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c357e8-fd8c-4eb2-b346-b6fc1723bd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.880951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c357e8-fd8c-4eb2-b346-b6fc1723bd9d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.883259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:07.892592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 987/1945] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:07.896423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c357e8-fd8c-4eb2-b346-b6fc1723bd9d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.905941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c357e8-fd8c-4eb2-b346-b6fc1723bd9d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.910556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826231d-f73e-4b9d-b890-98d8589590e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.911182 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826231d-f73e-4b9d-b890-98d8589590e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.911466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826231d-f73e-4b9d-b890-98d8589590e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.916870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826231d-f73e-4b9d-b890-98d8589590e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.916870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826231d-f73e-4b9d-b890-98d8589590e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.916870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826231d-f73e-4b9d-b890-98d8589590e5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.924388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 988/1946] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:07.928206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826231d-f73e-4b9d-b890-98d8589590e5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.939417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826231d-f73e-4b9d-b890-98d8589590e5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.944034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304b1a4-2499-4ce6-9015-e266094398eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.945734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304b1a4-2499-4ce6-9015-e266094398eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.946010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304b1a4-2499-4ce6-9015-e266094398eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.946308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304b1a4-2499-4ce6-9015-e266094398eb admin admin] PATH_INFO: `/v3/projects/d3670d2a04aa477abcdf974cf9d2def0/users/32c692563d814b37a40dd1b64a480167/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.962084 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304b1a4-2499-4ce6-9015-e266094398eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3670d2a04aa477abcdf974cf9d2def0, user_id=32c692563d814b37a40dd1b64a480167, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.964950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304b1a4-2499-4ce6-9015-e266094398eb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:07.973305 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 959/1947] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:07 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:07.974334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8577dca-6cd7-439c-b4f9-7159ceefe7d7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:07.988562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8577dca-6cd7-439c-b4f9-7159ceefe7d7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:07.988713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6688715-befd-447e-984f-75bef05a079d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:07.989341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6688715-befd-447e-984f-75bef05a079d admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:07.989624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6688715-befd-447e-984f-75bef05a079d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:07.989902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6688715-befd-447e-984f-75bef05a079d admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:07.990661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6688715-befd-447e-984f-75bef05a079d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:07.992276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6688715-befd-447e-984f-75bef05a079d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:08.002786 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 960/1948] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:08.006387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6688715-befd-447e-984f-75bef05a079d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:08.016558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6688715-befd-447e-984f-75bef05a079d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:08.021163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0853-51b5-418c-8501-227eab05fbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:08.021780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0853-51b5-418c-8501-227eab05fbeb admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.022095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0853-51b5-418c-8501-227eab05fbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.022376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0853-51b5-418c-8501-227eab05fbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.023076 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a0853-51b5-418c-8501-227eab05fbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:08.024658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a0853-51b5-418c-8501-227eab05fbeb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:08.035125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 961/1949] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:08.039131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0853-51b5-418c-8501-227eab05fbeb admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:08.048726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0853-51b5-418c-8501-227eab05fbeb admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:08.053756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b169b95-9a1d-4acc-b849-2ec63a214b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:08.054676 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b169b95-9a1d-4acc-b849-2ec63a214b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.054943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b169b95-9a1d-4acc-b849-2ec63a214b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.055244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b169b95-9a1d-4acc-b849-2ec63a214b56 admin admin] PATH_INFO: `/v3/projects/cac9ba9bdc3d412d82547f24dd96ea3a/users/ac992bc5bda944b2a62de3e0e12baafe/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.067535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 989/1950] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:07 2022] PUT /identity/v3/projects/d3670d2a04aa477abcdf974cf9d2def0/users/32c692563d814b37a40dd1b64a480167/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:08.077421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b169b95-9a1d-4acc-b849-2ec63a214b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac9ba9bdc3d412d82547f24dd96ea3a, user_id=ac992bc5bda944b2a62de3e0e12baafe, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:08.077421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b169b95-9a1d-4acc-b849-2ec63a214b56 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:08.080748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097633f-1690-4337-951d-fbe3837b65de None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.081093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097633f-1690-4337-951d-fbe3837b65de None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.081307 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097633f-1690-4337-951d-fbe3837b65de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.108850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a097633f-1690-4337-951d-fbe3837b65de None None] MFA Rules not processed for user `32c692563d814b37a40dd1b64a480167`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:08.110157 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a097633f-1690-4337-951d-fbe3837b65de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:08.156574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 962/1951] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:08 2022] PUT /identity/v3/projects/cac9ba9bdc3d412d82547f24dd96ea3a/users/ac992bc5bda944b2a62de3e0e12baafe/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:08.167923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bc9fd-f638-4a28-a970-f786211c335c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.168268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bc9fd-f638-4a28-a970-f786211c335c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.168486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bc9fd-f638-4a28-a970-f786211c335c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.199067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 990/1952] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:08.204032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097633f-1690-4337-951d-fbe3837b65de None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:08.212415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-042bc9fd-f638-4a28-a970-f786211c335c None None] MFA Rules not processed for user `ac992bc5bda944b2a62de3e0e12baafe`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:08.213747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-042bc9fd-f638-4a28-a970-f786211c335c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:08.278486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097633f-1690-4337-951d-fbe3837b65de None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:08.287023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:08.300931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 963/1953] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:08.301769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bc9fd-f638-4a28-a970-f786211c335c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:08.362433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429eed3-66d5-49de-898b-c52659b060df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:08.363322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429eed3-66d5-49de-898b-c52659b060df admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.363643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429eed3-66d5-49de-898b-c52659b060df admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.363911 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429eed3-66d5-49de-898b-c52659b060df admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.364636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429eed3-66d5-49de-898b-c52659b060df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:08.387345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429eed3-66d5-49de-898b-c52659b060df admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:08.393401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:08.409786 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bc9fd-f638-4a28-a970-f786211c335c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:08.421216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e252-8841-4ba4-9cef-1cb68cc2ca1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:08.422505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e252-8841-4ba4-9cef-1cb68cc2ca1d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.422830 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e252-8841-4ba4-9cef-1cb68cc2ca1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.423175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e252-8841-4ba4-9cef-1cb68cc2ca1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.423893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3e252-8841-4ba4-9cef-1cb68cc2ca1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:08.427142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3e252-8841-4ba4-9cef-1cb68cc2ca1d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:08.526155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 991/1954] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:08 2022] POST /identity/v3/projects => generated 377 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:08.527296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429eed3-66d5-49de-898b-c52659b060df admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:08.537722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 964/1955] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:08 2022] POST /identity/v3/projects => generated 367 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:08.538911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e252-8841-4ba4-9cef-1cb68cc2ca1d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:08.612828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429eed3-66d5-49de-898b-c52659b060df admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:08.675023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e252-8841-4ba4-9cef-1cb68cc2ca1d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:08.691239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a66e25-f835-4a53-9617-15738b1fa962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:08.692071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66e25-f835-4a53-9617-15738b1fa962 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.692468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66e25-f835-4a53-9617-15738b1fa962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.692815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66e25-f835-4a53-9617-15738b1fa962 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.694207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a66e25-f835-4a53-9617-15738b1fa962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:08.695839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a66e25-f835-4a53-9617-15738b1fa962 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:08.714863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb27172-0d94-41d7-bdd6-2e386ebcbd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:08.715543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb27172-0d94-41d7-bdd6-2e386ebcbd2f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.715825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb27172-0d94-41d7-bdd6-2e386ebcbd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.716103 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb27172-0d94-41d7-bdd6-2e386ebcbd2f admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.717198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb27172-0d94-41d7-bdd6-2e386ebcbd2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:08.719318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb27172-0d94-41d7-bdd6-2e386ebcbd2f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:08.804530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 965/1956] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:08 2022] POST /identity/v3/users => generated 290 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:08.808504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a66e25-f835-4a53-9617-15738b1fa962 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:08.824880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a66e25-f835-4a53-9617-15738b1fa962 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:08.825601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 992/1957] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:08 2022] POST /identity/v3/users => generated 295 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:08.831660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb4ac0-c1f5-4ba8-82d0-b26261ab4864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:08.832087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb27172-0d94-41d7-bdd6-2e386ebcbd2f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:08.832345 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb4ac0-c1f5-4ba8-82d0-b26261ab4864 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.832631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb4ac0-c1f5-4ba8-82d0-b26261ab4864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.832910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb4ac0-c1f5-4ba8-82d0-b26261ab4864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.833643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb4ac0-c1f5-4ba8-82d0-b26261ab4864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:08.836574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb4ac0-c1f5-4ba8-82d0-b26261ab4864 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:08.849434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 966/1958] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:08.853497 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb4ac0-c1f5-4ba8-82d0-b26261ab4864 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:08.857734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb27172-0d94-41d7-bdd6-2e386ebcbd2f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:08.872201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5cedc-83e5-4c63-8629-a11d4d0ef82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:08.872728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb4ac0-c1f5-4ba8-82d0-b26261ab4864 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:08.872907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5cedc-83e5-4c63-8629-a11d4d0ef82f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.873175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5cedc-83e5-4c63-8629-a11d4d0ef82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.873450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5cedc-83e5-4c63-8629-a11d4d0ef82f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.874299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5cedc-83e5-4c63-8629-a11d4d0ef82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:08.877129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5cedc-83e5-4c63-8629-a11d4d0ef82f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:08.880586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb4d4f-f3f5-4ef3-a6a8-c168ac34515e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:08.882077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb4d4f-f3f5-4ef3-a6a8-c168ac34515e admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.882077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb4d4f-f3f5-4ef3-a6a8-c168ac34515e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.882253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb4d4f-f3f5-4ef3-a6a8-c168ac34515e admin admin] PATH_INFO: `/v3/projects/a689658abe0c4b61a00ccc3419e5e7fe/users/14ffef5633994cc19d9530d169a99383/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.888057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 993/1959] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:08.893387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5cedc-83e5-4c63-8629-a11d4d0ef82f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:08.901200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb4d4f-f3f5-4ef3-a6a8-c168ac34515e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a689658abe0c4b61a00ccc3419e5e7fe, user_id=14ffef5633994cc19d9530d169a99383, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:08.904083 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb4d4f-f3f5-4ef3-a6a8-c168ac34515e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:08.910392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5cedc-83e5-4c63-8629-a11d4d0ef82f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:08.915123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b7abd-e23d-4f1a-84c9-6a279961cfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:08.916013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b7abd-e23d-4f1a-84c9-6a279961cfac admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:08.916305 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b7abd-e23d-4f1a-84c9-6a279961cfac admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:08.916563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b7abd-e23d-4f1a-84c9-6a279961cfac admin admin] PATH_INFO: `/v3/projects/5042e05651f84f73b4a2eb63a066f72b/users/fbebfe75bcc2418eb9f1f82143b71ff4/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:08.932380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b7abd-e23d-4f1a-84c9-6a279961cfac admin admin] RBAC: Authorizing `identity:create_grant(project_id=5042e05651f84f73b4a2eb63a066f72b, user_id=fbebfe75bcc2418eb9f1f82143b71ff4, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:08.935020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b7abd-e23d-4f1a-84c9-6a279961cfac admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:09.004055 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 994/1960] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:08 2022] PUT /identity/v3/projects/5042e05651f84f73b4a2eb63a066f72b/users/fbebfe75bcc2418eb9f1f82143b71ff4/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:09.007968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b7abd-e23d-4f1a-84c9-6a279961cfac admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:09.009484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 967/1961] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:08 2022] PUT /identity/v3/projects/a689658abe0c4b61a00ccc3419e5e7fe/users/14ffef5633994cc19d9530d169a99383/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:09.014098 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb4d4f-f3f5-4ef3-a6a8-c168ac34515e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:09.094160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:09.108197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:09.143009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b7abd-e23d-4f1a-84c9-6a279961cfac admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:09.145437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb4d4f-f3f5-4ef3-a6a8-c168ac34515e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:09.150574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cac1c0-2714-49a7-87b8-b03f51649ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:09.151513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cac1c0-2714-49a7-87b8-b03f51649ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:09.152009 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cac1c0-2714-49a7-87b8-b03f51649ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:09.152494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cac1c0-2714-49a7-87b8-b03f51649ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:09.153453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cac1c0-2714-49a7-87b8-b03f51649ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:09.155312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cac1c0-2714-49a7-87b8-b03f51649ee1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:09.156186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2dbe4-e230-4922-bae7-88cf51d8dc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:09.157704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2dbe4-e230-4922-bae7-88cf51d8dc46 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:09.158063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2dbe4-e230-4922-bae7-88cf51d8dc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:09.158340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2dbe4-e230-4922-bae7-88cf51d8dc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:09.159615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2dbe4-e230-4922-bae7-88cf51d8dc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:09.161580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2dbe4-e230-4922-bae7-88cf51d8dc46 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:09.169142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 968/1962] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:09.177210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cac1c0-2714-49a7-87b8-b03f51649ee1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:09.182166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 995/1963] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:09.183712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2dbe4-e230-4922-bae7-88cf51d8dc46 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:09.191879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cac1c0-2714-49a7-87b8-b03f51649ee1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:09.197944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e553697-bb9e-48ea-9b9a-a865c47932d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:09.198793 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2dbe4-e230-4922-bae7-88cf51d8dc46 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:09.200039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e553697-bb9e-48ea-9b9a-a865c47932d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:09.200687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e553697-bb9e-48ea-9b9a-a865c47932d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:09.201145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e553697-bb9e-48ea-9b9a-a865c47932d8 admin admin] PATH_INFO: `/v3/projects/a689658abe0c4b61a00ccc3419e5e7fe/users/14ffef5633994cc19d9530d169a99383/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:09.205700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9325bcd-86f8-47fc-a999-3f70af522ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:09.207603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e553697-bb9e-48ea-9b9a-a865c47932d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a689658abe0c4b61a00ccc3419e5e7fe, user_id=14ffef5633994cc19d9530d169a99383, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:09.207710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325bcd-86f8-47fc-a999-3f70af522ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:09.207710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325bcd-86f8-47fc-a999-3f70af522ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:09.207710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325bcd-86f8-47fc-a999-3f70af522ae6 admin admin] PATH_INFO: `/v3/projects/5042e05651f84f73b4a2eb63a066f72b/users/fbebfe75bcc2418eb9f1f82143b71ff4/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:09.209854 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e553697-bb9e-48ea-9b9a-a865c47932d8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:09.212260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9325bcd-86f8-47fc-a999-3f70af522ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5042e05651f84f73b4a2eb63a066f72b, user_id=fbebfe75bcc2418eb9f1f82143b71ff4, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:09.215545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9325bcd-86f8-47fc-a999-3f70af522ae6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:09.221427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 969/1964] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:09 2022] PUT /identity/v3/projects/a689658abe0c4b61a00ccc3419e5e7fe/users/14ffef5633994cc19d9530d169a99383/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:09.228913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 996/1965] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:09 2022] PUT /identity/v3/projects/5042e05651f84f73b4a2eb63a066f72b/users/fbebfe75bcc2418eb9f1f82143b71ff4/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:09.236311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb324ac6-93b8-4be5-bcdd-7620ffd61e59 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:09.236745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb324ac6-93b8-4be5-bcdd-7620ffd61e59 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:09.237131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb324ac6-93b8-4be5-bcdd-7620ffd61e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:09.243162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcdffc-2524-438e-95b3-f330ebdb415e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:09.253837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcdffc-2524-438e-95b3-f330ebdb415e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:09.253837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcdffc-2524-438e-95b3-f330ebdb415e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:09.274899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bb324ac6-93b8-4be5-bcdd-7620ffd61e59 None None] MFA Rules not processed for user `14ffef5633994cc19d9530d169a99383`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:09.276126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bb324ac6-93b8-4be5-bcdd-7620ffd61e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:09.277424 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-14fcdffc-2524-438e-95b3-f330ebdb415e None None] MFA Rules not processed for user `fbebfe75bcc2418eb9f1f82143b71ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:09.278983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-14fcdffc-2524-438e-95b3-f330ebdb415e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:09.362792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 997/1966] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:09.374494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcdffc-2524-438e-95b3-f330ebdb415e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:09.377251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-14fcdffc-2524-438e-95b3-f330ebdb415e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:09.387691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 970/1967] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:09.445203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcdffc-2524-438e-95b3-f330ebdb415e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:09.448743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981dfe4-5332-4370-8c13-e3d7582796fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:09.449394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981dfe4-5332-4370-8c13-e3d7582796fa service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:09.449676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981dfe4-5332-4370-8c13-e3d7582796fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:09.449951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981dfe4-5332-4370-8c13-e3d7582796fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:09.451823 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0981dfe4-5332-4370-8c13-e3d7582796fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:09.521464 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981dfe4-5332-4370-8c13-e3d7582796fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:09.522952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981dfe4-5332-4370-8c13-e3d7582796fa service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:09.532526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 998/1968] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:07:09.628590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb324ac6-93b8-4be5-bcdd-7620ffd61e59 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:09.629985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bb324ac6-93b8-4be5-bcdd-7620ffd61e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:09.643708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb324ac6-93b8-4be5-bcdd-7620ffd61e59 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:09.647209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c76657-ea57-48e4-9f58-a8fb40923434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:09.647954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c76657-ea57-48e4-9f58-a8fb40923434 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:09.648116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c76657-ea57-48e4-9f58-a8fb40923434 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:09.648393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c76657-ea57-48e4-9f58-a8fb40923434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:09.659660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c76657-ea57-48e4-9f58-a8fb40923434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:09.661330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c76657-ea57-48e4-9f58-a8fb40923434 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:09.672941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 971/1969] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:12.767574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981dfe4-5332-4370-8c13-e3d7582796fa service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:12.778190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:12.846336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981dfe4-5332-4370-8c13-e3d7582796fa service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:12.851518 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:12.852449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:12.852733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:12.853013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] PATH_INFO: `/v3/users/ac992bc5bda944b2a62de3e0e12baafe` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:12.854241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac992bc5bda944b2a62de3e0e12baafe)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:12.856206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:13.052583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac992bc5bda944b2a62de3e0e12baafe'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:13.082754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac992bc5bda944b2a62de3e0e12baafe'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:13.158108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac992bc5bda944b2a62de3e0e12baafe'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:13.158831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac992bc5bda944b2a62de3e0e12baafe'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:13.159514 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac992bc5bda944b2a62de3e0e12baafe'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:13.178793 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 999/1970] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:12 2022] DELETE /identity/v3/users/ac992bc5bda944b2a62de3e0e12baafe => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:13.183638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efe601-242d-4e86-b5be-30b944507567 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:13.184149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efe601-242d-4e86-b5be-30b944507567 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:13.184417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efe601-242d-4e86-b5be-30b944507567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:13.205262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-87efe601-242d-4e86-b5be-30b944507567 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:13.206734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-87efe601-242d-4e86-b5be-30b944507567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:13.272399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 972/1971] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:13.282206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:13.283682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:13.353599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8438f9d-3119-4653-a586-beeb36b503f5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:13.357418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958eeeb-ea03-4cf0-8eba-c010fdd9e8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:13.358258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958eeeb-ea03-4cf0-8eba-c010fdd9e8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:13.359599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958eeeb-ea03-4cf0-8eba-c010fdd9e8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:13.359599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958eeeb-ea03-4cf0-8eba-c010fdd9e8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:13.370570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958eeeb-ea03-4cf0-8eba-c010fdd9e8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:13.372111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958eeeb-ea03-4cf0-8eba-c010fdd9e8a2 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:13.385449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1000/1972] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:13.905863 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efe601-242d-4e86-b5be-30b944507567 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:13.907368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-87efe601-242d-4e86-b5be-30b944507567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:13.943273 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958eeeb-ea03-4cf0-8eba-c010fdd9e8a2 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:13.943749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:13.947780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b958eeeb-ea03-4cf0-8eba-c010fdd9e8a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:13.960335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958eeeb-ea03-4cf0-8eba-c010fdd9e8a2 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:13.965797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:13.966888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:13.969439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:13.970919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] PATH_INFO: `/v3/projects/cac9ba9bdc3d412d82547f24dd96ea3a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:13.974218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efe601-242d-4e86-b5be-30b944507567 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:13.977106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cac9ba9bdc3d412d82547f24dd96ea3a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:13.978906 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701214b5-3563-4fa4-a7e4-e73b570c32f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:13.979081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:13.979712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701214b5-3563-4fa4-a7e4-e73b570c32f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:13.980053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701214b5-3563-4fa4-a7e4-e73b570c32f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:13.980394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701214b5-3563-4fa4-a7e4-e73b570c32f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:13.983424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-701214b5-3563-4fa4-a7e4-e73b570c32f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:14.052181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701214b5-3563-4fa4-a7e4-e73b570c32f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:14.053757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701214b5-3563-4fa4-a7e4-e73b570c32f2 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:14.063073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 973/1973] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:14.097550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:14.119620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac9ba9bdc3d412d82547f24dd96ea3a', 'request_id': 'req-2f5523e7-29c9-4171-947e-081b19c59e97'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.128842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac9ba9bdc3d412d82547f24dd96ea3a', 'request_id': 'req-2f5523e7-29c9-4171-947e-081b19c59e97'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.136273 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac9ba9bdc3d412d82547f24dd96ea3a', 'request_id': 'req-2f5523e7-29c9-4171-947e-081b19c59e97'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.225951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] The token cache is being invalidate because project cac9ba9bdc3d412d82547f24dd96ea3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:14.226271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.226900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.228481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1001/1974] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:13 2022] DELETE /identity/v3/projects/cac9ba9bdc3d412d82547f24dd96ea3a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:14.232520 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701214b5-3563-4fa4-a7e4-e73b570c32f2 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:14.243415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701214b5-3563-4fa4-a7e4-e73b570c32f2 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:14.299071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:14.299980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:14.300260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:14.300527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] PATH_INFO: `/v3/users/14ffef5633994cc19d9530d169a99383` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:14.301716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ffef5633994cc19d9530d169a99383)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:14.303814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:14.580570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ffef5633994cc19d9530d169a99383'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.588476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ffef5633994cc19d9530d169a99383'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.602401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:14.611943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:14.617053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ffef5633994cc19d9530d169a99383'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.634581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5523e7-29c9-4171-947e-081b19c59e97 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:14.640368 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:14.642011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:14.642522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:14.643448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] PATH_INFO: `/v3/users/f824a504d04c4abcb9360db5a5962a5b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:14.652807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f824a504d04c4abcb9360db5a5962a5b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:14.654804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:14.701790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ffef5633994cc19d9530d169a99383'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.701790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6fed44a3-e540-4349-8eb9-f52cdb5a2665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ffef5633994cc19d9530d169a99383'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.703116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 974/1975] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:14 2022] DELETE /identity/v3/users/14ffef5633994cc19d9530d169a99383 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:14.948721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f824a504d04c4abcb9360db5a5962a5b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:14.956858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f824a504d04c4abcb9360db5a5962a5b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:15.038339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f824a504d04c4abcb9360db5a5962a5b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:15.039037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f824a504d04c4abcb9360db5a5962a5b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:15.040564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f824a504d04c4abcb9360db5a5962a5b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:15.063213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1002/1976] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:14 2022] DELETE /identity/v3/users/f824a504d04c4abcb9360db5a5962a5b => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:15.068423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fabff-e49b-401b-ada1-28f4904a0904 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:15.068719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fabff-e49b-401b-ada1-28f4904a0904 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:15.069058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fabff-e49b-401b-ada1-28f4904a0904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:15.078754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:15.091312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-719fabff-e49b-401b-ada1-28f4904a0904 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:15.092538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-719fabff-e49b-401b-ada1-28f4904a0904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:15.156686 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 975/1977] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:15.168004 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:15.169425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:15.239155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f4c13-75ab-4e61-adcf-7029d8c3f2e7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:15.242869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3af9ee-a7ab-4a9c-a581-783eb0a6be1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:15.243520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3af9ee-a7ab-4a9c-a581-783eb0a6be1d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:15.243959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3af9ee-a7ab-4a9c-a581-783eb0a6be1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:15.244207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3af9ee-a7ab-4a9c-a581-783eb0a6be1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:15.254521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3af9ee-a7ab-4a9c-a581-783eb0a6be1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:15.256214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3af9ee-a7ab-4a9c-a581-783eb0a6be1d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:15.266778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1003/1978] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:15.327186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fabff-e49b-401b-ada1-28f4904a0904 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:15.329408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-719fabff-e49b-401b-ada1-28f4904a0904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:15.363331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fabff-e49b-401b-ada1-28f4904a0904 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:15.370701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:15.372026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:15.372771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:15.372866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] PATH_INFO: `/v3/projects/a689658abe0c4b61a00ccc3419e5e7fe` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:15.375457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a689658abe0c4b61a00ccc3419e5e7fe)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:15.378711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:15.540362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a689658abe0c4b61a00ccc3419e5e7fe', 'request_id': 'req-29c6feb1-5280-4ef6-88d7-752802e4c7cb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:15.546448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a689658abe0c4b61a00ccc3419e5e7fe', 'request_id': 'req-29c6feb1-5280-4ef6-88d7-752802e4c7cb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:15.636645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a689658abe0c4b61a00ccc3419e5e7fe', 'request_id': 'req-29c6feb1-5280-4ef6-88d7-752802e4c7cb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:15.649114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] The token cache is being invalidate because project a689658abe0c4b61a00ccc3419e5e7fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:15.649492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:15.650245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29c6feb1-5280-4ef6-88d7-752802e4c7cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:15.651810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 976/1979] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:15 2022] DELETE /identity/v3/projects/a689658abe0c4b61a00ccc3419e5e7fe => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:15.661321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eaa25-540e-4524-bc7b-9995901994db None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:15.661321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eaa25-540e-4524-bc7b-9995901994db None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:15.661321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eaa25-540e-4524-bc7b-9995901994db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:15.682309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a74eaa25-540e-4524-bc7b-9995901994db None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:15.683856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a74eaa25-540e-4524-bc7b-9995901994db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:15.727421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:15.769064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1004/1980] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:15.778238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05e279-6e98-4d8d-9878-85c3ebaf3e80 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:15.778526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05e279-6e98-4d8d-9878-85c3ebaf3e80 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:15.778804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05e279-6e98-4d8d-9878-85c3ebaf3e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:15.798722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bf05e279-6e98-4d8d-9878-85c3ebaf3e80 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:15.801400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bf05e279-6e98-4d8d-9878-85c3ebaf3e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:15.811249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 977/1981] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:15.812066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74eaa25-540e-4524-bc7b-9995901994db None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:15.812203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05e279-6e98-4d8d-9878-85c3ebaf3e80 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:15.814644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a74eaa25-540e-4524-bc7b-9995901994db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:15.815246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bf05e279-6e98-4d8d-9878-85c3ebaf3e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:15.827555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05e279-6e98-4d8d-9878-85c3ebaf3e80 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:15.833557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:15.834759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:15.835518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:15.835902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] PATH_INFO: `/v3/projects/bdc3e6f1740445f2884bafb487d27906` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:15.837474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc3e6f1740445f2884bafb487d27906)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:15.839844 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:15.889779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74eaa25-540e-4524-bc7b-9995901994db None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:15.893674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388b9e1-4599-4b64-9154-d7b7a84ca6ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:15.894352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388b9e1-4599-4b64-9154-d7b7a84ca6ba service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:15.894730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388b9e1-4599-4b64-9154-d7b7a84ca6ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:15.895011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388b9e1-4599-4b64-9154-d7b7a84ca6ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:15.896868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b388b9e1-4599-4b64-9154-d7b7a84ca6ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:15.958631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388b9e1-4599-4b64-9154-d7b7a84ca6ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:15.960109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388b9e1-4599-4b64-9154-d7b7a84ca6ba service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:15.970135 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1005/1982] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:07:15.974164 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388b9e1-4599-4b64-9154-d7b7a84ca6ba service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:15.976261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc3e6f1740445f2884bafb487d27906', 'request_id': 'req-d241d814-5a78-47c4-aad7-043326f1e6ba'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:15.981352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388b9e1-4599-4b64-9154-d7b7a84ca6ba service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:15.984581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc3e6f1740445f2884bafb487d27906', 'request_id': 'req-d241d814-5a78-47c4-aad7-043326f1e6ba'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:16.037058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd002449-b3fe-48e5-ae68-8da65843eab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:16.037719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd002449-b3fe-48e5-ae68-8da65843eab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:16.038003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd002449-b3fe-48e5-ae68-8da65843eab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:16.038307 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd002449-b3fe-48e5-ae68-8da65843eab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:16.039025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd002449-b3fe-48e5-ae68-8da65843eab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:16.039330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd002449-b3fe-48e5-ae68-8da65843eab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:16.040917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd002449-b3fe-48e5-ae68-8da65843eab6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:16.052795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1006/1983] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:07:16.053791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc3e6f1740445f2884bafb487d27906', 'request_id': 'req-d241d814-5a78-47c4-aad7-043326f1e6ba'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:16.056985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd002449-b3fe-48e5-ae68-8da65843eab6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:16.065592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] The token cache is being invalidate because project bdc3e6f1740445f2884bafb487d27906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:16.066412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:16.066642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd002449-b3fe-48e5-ae68-8da65843eab6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:16.067135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:16.068864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 978/1984] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:15 2022] DELETE /identity/v3/projects/bdc3e6f1740445f2884bafb487d27906 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:16.071334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb67519-edff-449e-a516-bbb651566079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:16.071978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb67519-edff-449e-a516-bbb651566079 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:16.072262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb67519-edff-449e-a516-bbb651566079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:16.072538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb67519-edff-449e-a516-bbb651566079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:16.072959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:16.073213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb67519-edff-449e-a516-bbb651566079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:16.074586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:16.075146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb67519-edff-449e-a516-bbb651566079 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:16.084833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:16.088793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241d814-5a78-47c4-aad7-043326f1e6ba admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:16.094862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:16.097434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:16.097434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:16.097434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] PATH_INFO: `/v3/users/aafddfa737124229b77dd87fdd3d6c22` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:16.098464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aafddfa737124229b77dd87fdd3d6c22)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:16.100237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:16.181200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1007/1985] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:16 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:16.185370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb67519-edff-449e-a516-bbb651566079 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:16.186993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ecb67519-edff-449e-a516-bbb651566079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:16.256670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb67519-edff-449e-a516-bbb651566079 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:16.261407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd77a60-abc8-4249-9f67-319743fc775e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:16.262093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd77a60-abc8-4249-9f67-319743fc775e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:16.262376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd77a60-abc8-4249-9f67-319743fc775e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:16.262648 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd77a60-abc8-4249-9f67-319743fc775e admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:16.263383 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd77a60-abc8-4249-9f67-319743fc775e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:16.265279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd77a60-abc8-4249-9f67-319743fc775e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:16.329412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aafddfa737124229b77dd87fdd3d6c22'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:16.336959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aafddfa737124229b77dd87fdd3d6c22'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:16.349569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aafddfa737124229b77dd87fdd3d6c22'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:16.374012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1008/1986] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:16 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:16.378251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd77a60-abc8-4249-9f67-319743fc775e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:16.388072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd77a60-abc8-4249-9f67-319743fc775e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:16.393058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c07003-d044-47dc-8667-002bde9273d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:16.394396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c07003-d044-47dc-8667-002bde9273d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:16.394717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c07003-d044-47dc-8667-002bde9273d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:16.395005 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c07003-d044-47dc-8667-002bde9273d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:16.395738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c07003-d044-47dc-8667-002bde9273d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:16.397338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c07003-d044-47dc-8667-002bde9273d1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:16.407354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1009/1987] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:16.414923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c07003-d044-47dc-8667-002bde9273d1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:16.424588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c07003-d044-47dc-8667-002bde9273d1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:16.429540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5572a0-e15e-45b7-a467-25eb99f673ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:16.430214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5572a0-e15e-45b7-a467-25eb99f673ed admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:16.430503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5572a0-e15e-45b7-a467-25eb99f673ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:16.430783 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5572a0-e15e-45b7-a467-25eb99f673ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:16.431541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5572a0-e15e-45b7-a467-25eb99f673ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:16.433096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5572a0-e15e-45b7-a467-25eb99f673ed admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:16.436536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aafddfa737124229b77dd87fdd3d6c22'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:16.437281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e185a142-8506-48d7-abfe-a335004def08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aafddfa737124229b77dd87fdd3d6c22'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:16.445619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1010/1988] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:16.445782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 979/1989] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:16 2022] DELETE /identity/v3/users/aafddfa737124229b77dd87fdd3d6c22 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:16.449602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5572a0-e15e-45b7-a467-25eb99f673ed admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:16.451316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3c5572a0-e15e-45b7-a467-25eb99f673ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:16.464627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5572a0-e15e-45b7-a467-25eb99f673ed admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:16.467717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:16.527608 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb82fe-5136-4588-8a3f-4a23f447a269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:16.528500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb82fe-5136-4588-8a3f-4a23f447a269 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:16.528783 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb82fe-5136-4588-8a3f-4a23f447a269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:16.529062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb82fe-5136-4588-8a3f-4a23f447a269 admin admin] PATH_INFO: `/v3/projects/8bf5884aaeab4c3bbe6521e634553bcb/users/a8a8e30d822f4c1689c29934f00377b2/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:16.543741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb82fe-5136-4588-8a3f-4a23f447a269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf5884aaeab4c3bbe6521e634553bcb, user_id=a8a8e30d822f4c1689c29934f00377b2, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:16.546625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb82fe-5136-4588-8a3f-4a23f447a269 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:16.645923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1011/1990] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:16 2022] PUT /identity/v3/projects/8bf5884aaeab4c3bbe6521e634553bcb/users/a8a8e30d822f4c1689c29934f00377b2/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:16.656792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82268f-7ea4-4ce2-a7f3-50ad26fae7ec None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:16.657080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82268f-7ea4-4ce2-a7f3-50ad26fae7ec None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:16.657357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82268f-7ea4-4ce2-a7f3-50ad26fae7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:16.686731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9f82268f-7ea4-4ce2-a7f3-50ad26fae7ec None None] MFA Rules not processed for user `a8a8e30d822f4c1689c29934f00377b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:16.687910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9f82268f-7ea4-4ce2-a7f3-50ad26fae7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:16.755695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 980/1991] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:16.760001 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb82fe-5136-4588-8a3f-4a23f447a269 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:16.815462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:16.833263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb82fe-5136-4588-8a3f-4a23f447a269 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:16.838062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80f037-0daf-4c30-8050-f5187969492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:16.838724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80f037-0daf-4c30-8050-f5187969492b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:16.839034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80f037-0daf-4c30-8050-f5187969492b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:16.839312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80f037-0daf-4c30-8050-f5187969492b admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:16.840002 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80f037-0daf-4c30-8050-f5187969492b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:16.843596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80f037-0daf-4c30-8050-f5187969492b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:16.867872 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82268f-7ea4-4ce2-a7f3-50ad26fae7ec None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:16.869319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9f82268f-7ea4-4ce2-a7f3-50ad26fae7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:16.883535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82268f-7ea4-4ce2-a7f3-50ad26fae7ec None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:16.888090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:16.888971 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:16.889783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:16.890097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] PATH_INFO: `/v3/projects/b8c94d2924364e189aa4f1b989008f74` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:16.891565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8c94d2924364e189aa4f1b989008f74)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:16.894007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:16.931279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1012/1992] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:16 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:16.935541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80f037-0daf-4c30-8050-f5187969492b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:16.998720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80f037-0daf-4c30-8050-f5187969492b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.003580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c17ee-2b41-4789-8318-e89a6e4bcaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.004251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c17ee-2b41-4789-8318-e89a6e4bcaef admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.004556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c17ee-2b41-4789-8318-e89a6e4bcaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.004833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c17ee-2b41-4789-8318-e89a6e4bcaef admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.005523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c17ee-2b41-4789-8318-e89a6e4bcaef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.007633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c17ee-2b41-4789-8318-e89a6e4bcaef admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.036272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8c94d2924364e189aa4f1b989008f74', 'request_id': 'req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:17.042379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8c94d2924364e189aa4f1b989008f74', 'request_id': 'req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:17.045042 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:17.118572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1013/1993] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:16 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:17.122713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c17ee-2b41-4789-8318-e89a6e4bcaef admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.123694 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8c94d2924364e189aa4f1b989008f74', 'request_id': 'req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:17.132594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c17ee-2b41-4789-8318-e89a6e4bcaef admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.135836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] The token cache is being invalidate because project b8c94d2924364e189aa4f1b989008f74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:17.136125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:17.137023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d370d3bb-0840-4c05-ad8c-2fb3a27a5ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:17.139084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 981/1994] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:16 2022] DELETE /identity/v3/projects/b8c94d2924364e189aa4f1b989008f74 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:17.148214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f962ce6-8b2b-4097-8494-29c50a79dc0a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.148502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f962ce6-8b2b-4097-8494-29c50a79dc0a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.148795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f962ce6-8b2b-4097-8494-29c50a79dc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.168631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4f962ce6-8b2b-4097-8494-29c50a79dc0a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:17.169852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4f962ce6-8b2b-4097-8494-29c50a79dc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:17.194314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d827db-4bfb-4eff-95b4-0cd8fe5478ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.194989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d827db-4bfb-4eff-95b4-0cd8fe5478ea admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.195274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d827db-4bfb-4eff-95b4-0cd8fe5478ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.195554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d827db-4bfb-4eff-95b4-0cd8fe5478ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.196287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d827db-4bfb-4eff-95b4-0cd8fe5478ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.197907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d827db-4bfb-4eff-95b4-0cd8fe5478ea admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.209016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1014/1995] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:17.213114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d827db-4bfb-4eff-95b4-0cd8fe5478ea admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.214829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b6d827db-4bfb-4eff-95b4-0cd8fe5478ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:17.227078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d827db-4bfb-4eff-95b4-0cd8fe5478ea admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.230335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 982/1996] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:17.232061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97b027-7ec9-4e10-9bdf-82238266c039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.232935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97b027-7ec9-4e10-9bdf-82238266c039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.233215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97b027-7ec9-4e10-9bdf-82238266c039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.233489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97b027-7ec9-4e10-9bdf-82238266c039 admin admin] PATH_INFO: `/v3/projects/e820363244934a6f8e4b133dc256936d/users/5ab25b2ec50844539f4680ea00156869/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.237852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f2ec9-b202-42ba-966b-90cb0929b36e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.238172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f2ec9-b202-42ba-966b-90cb0929b36e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.238455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f2ec9-b202-42ba-966b-90cb0929b36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.248656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97b027-7ec9-4e10-9bdf-82238266c039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e820363244934a6f8e4b133dc256936d, user_id=5ab25b2ec50844539f4680ea00156869, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.251421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97b027-7ec9-4e10-9bdf-82238266c039 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.257230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5a3f2ec9-b202-42ba-966b-90cb0929b36e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:17.258469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5a3f2ec9-b202-42ba-966b-90cb0929b36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:17.266694 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 983/1997] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:17.271387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f2ec9-b202-42ba-966b-90cb0929b36e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.280479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f2ec9-b202-42ba-966b-90cb0929b36e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.285392 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc88e23-518e-44fd-9fa3-0807f3a6b9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.286076 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc88e23-518e-44fd-9fa3-0807f3a6b9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.286371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc88e23-518e-44fd-9fa3-0807f3a6b9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.286645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc88e23-518e-44fd-9fa3-0807f3a6b9c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.287370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc88e23-518e-44fd-9fa3-0807f3a6b9c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:17.287703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc88e23-518e-44fd-9fa3-0807f3a6b9c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.289303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc88e23-518e-44fd-9fa3-0807f3a6b9c0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.305434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 984/1998] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:07:17.321347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc88e23-518e-44fd-9fa3-0807f3a6b9c0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.321347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc88e23-518e-44fd-9fa3-0807f3a6b9c0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.321347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fc621-2389-47c8-9137-11173146532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.321992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fc621-2389-47c8-9137-11173146532c admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.321992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fc621-2389-47c8-9137-11173146532c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.322229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fc621-2389-47c8-9137-11173146532c admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.322908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fc621-2389-47c8-9137-11173146532c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.324768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fc621-2389-47c8-9137-11173146532c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.340381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1015/1999] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:17 2022] PUT /identity/v3/projects/e820363244934a6f8e4b133dc256936d/users/5ab25b2ec50844539f4680ea00156869/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:17.351093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97b027-7ec9-4e10-9bdf-82238266c039 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.382201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:17.413042 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97b027-7ec9-4e10-9bdf-82238266c039 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.417727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b701375-6b82-4195-9b7e-2e3ead879212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.418385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b701375-6b82-4195-9b7e-2e3ead879212 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.418671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b701375-6b82-4195-9b7e-2e3ead879212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.418950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b701375-6b82-4195-9b7e-2e3ead879212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.419850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b701375-6b82-4195-9b7e-2e3ead879212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.421309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b701375-6b82-4195-9b7e-2e3ead879212 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.427096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 985/2000] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:17 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:17.431455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fc621-2389-47c8-9137-11173146532c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.436543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1016/2001] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:17.437503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b701375-6b82-4195-9b7e-2e3ead879212 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.447786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b701375-6b82-4195-9b7e-2e3ead879212 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.497623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fc621-2389-47c8-9137-11173146532c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.503778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa870ba-11d3-486a-b140-d0c9add14f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.504496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa870ba-11d3-486a-b140-d0c9add14f50 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.504804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa870ba-11d3-486a-b140-d0c9add14f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.505077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa870ba-11d3-486a-b140-d0c9add14f50 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.505760 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa870ba-11d3-486a-b140-d0c9add14f50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.505864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5521cd-ae10-41b2-a662-6a9a6f86139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.506743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5521cd-ae10-41b2-a662-6a9a6f86139d admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.507035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5521cd-ae10-41b2-a662-6a9a6f86139d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.507312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5521cd-ae10-41b2-a662-6a9a6f86139d admin admin] PATH_INFO: `/v3/projects/e820363244934a6f8e4b133dc256936d/users/5ab25b2ec50844539f4680ea00156869/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.507766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa870ba-11d3-486a-b140-d0c9add14f50 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.509279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5521cd-ae10-41b2-a662-6a9a6f86139d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e820363244934a6f8e4b133dc256936d, user_id=5ab25b2ec50844539f4680ea00156869, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.511952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5521cd-ae10-41b2-a662-6a9a6f86139d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.523432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1017/2002] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:17 2022] PUT /identity/v3/projects/e820363244934a6f8e4b133dc256936d/users/5ab25b2ec50844539f4680ea00156869/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:17.533632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbf2c1-0e24-4392-a525-109d01631046 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.533979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbf2c1-0e24-4392-a525-109d01631046 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.534907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbf2c1-0e24-4392-a525-109d01631046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.562395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-aadbf2c1-0e24-4392-a525-109d01631046 None None] MFA Rules not processed for user `5ab25b2ec50844539f4680ea00156869`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:17.563597 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-aadbf2c1-0e24-4392-a525-109d01631046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:17.585944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 986/2003] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:17 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:17.591919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa870ba-11d3-486a-b140-d0c9add14f50 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.602544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa870ba-11d3-486a-b140-d0c9add14f50 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.636227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:17.636593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1018/2004] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:17.648340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbf2c1-0e24-4392-a525-109d01631046 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.649775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-aadbf2c1-0e24-4392-a525-109d01631046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:17.664193 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2b899-d5b4-4d68-a7d6-3ce6537a27aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.664875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2b899-d5b4-4d68-a7d6-3ce6537a27aa admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.665158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2b899-d5b4-4d68-a7d6-3ce6537a27aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.665435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2b899-d5b4-4d68-a7d6-3ce6537a27aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.666365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2b899-d5b4-4d68-a7d6-3ce6537a27aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.667955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2b899-d5b4-4d68-a7d6-3ce6537a27aa admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.678229 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 987/2005] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:17.681758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2b899-d5b4-4d68-a7d6-3ce6537a27aa admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.692172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2b899-d5b4-4d68-a7d6-3ce6537a27aa admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.696805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a0e5b-07ec-45e7-a398-ca2ca253b2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.697402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:17.697578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a0e5b-07ec-45e7-a398-ca2ca253b2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.697764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a0e5b-07ec-45e7-a398-ca2ca253b2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.698068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a0e5b-07ec-45e7-a398-ca2ca253b2a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.698778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a0e5b-07ec-45e7-a398-ca2ca253b2a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.700355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a0e5b-07ec-45e7-a398-ca2ca253b2a1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.708673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbf2c1-0e24-4392-a525-109d01631046 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.711796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 988/2006] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:17.714661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bec7c0-2327-4af1-a1f0-14e8f4432d39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.715476 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bec7c0-2327-4af1-a1f0-14e8f4432d39 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.715758 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bec7c0-2327-4af1-a1f0-14e8f4432d39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.715874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a0e5b-07ec-45e7-a398-ca2ca253b2a1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.716031 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bec7c0-2327-4af1-a1f0-14e8f4432d39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.726106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a0e5b-07ec-45e7-a398-ca2ca253b2a1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.726276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bec7c0-2327-4af1-a1f0-14e8f4432d39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.727584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bec7c0-2327-4af1-a1f0-14e8f4432d39 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.730779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343f6da-a203-44ba-aa38-8a43a60ed83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.731642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343f6da-a203-44ba-aa38-8a43a60ed83f admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.731924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343f6da-a203-44ba-aa38-8a43a60ed83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.732358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343f6da-a203-44ba-aa38-8a43a60ed83f admin admin] PATH_INFO: `/v3/projects/3898a3ddbb114254b3cd132bfe55ffe0/users/adea65d14bd748278f72a8c0e27fc282/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.738815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1019/2007] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:07:17.747840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343f6da-a203-44ba-aa38-8a43a60ed83f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3898a3ddbb114254b3cd132bfe55ffe0, user_id=adea65d14bd748278f72a8c0e27fc282, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:17.750580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343f6da-a203-44ba-aa38-8a43a60ed83f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:17.820725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 989/2008] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:17 2022] PUT /identity/v3/projects/3898a3ddbb114254b3cd132bfe55ffe0/users/adea65d14bd748278f72a8c0e27fc282/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:17.830742 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298c718-652b-412a-a919-0d7b01dc7395 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.831033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298c718-652b-412a-a919-0d7b01dc7395 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.831429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298c718-652b-412a-a919-0d7b01dc7395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:17.858487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7298c718-652b-412a-a919-0d7b01dc7395 None None] MFA Rules not processed for user `adea65d14bd748278f72a8c0e27fc282`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:17.859571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7298c718-652b-412a-a919-0d7b01dc7395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:17.927706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1020/2009] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:17.932128 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343f6da-a203-44ba-aa38-8a43a60ed83f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:17.993693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343f6da-a203-44ba-aa38-8a43a60ed83f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:17.998556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe4769-53b4-4b49-ad3f-4b953b09539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:17.999145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe4769-53b4-4b49-ad3f-4b953b09539f admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:17.999443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe4769-53b4-4b49-ad3f-4b953b09539f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:17.999702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe4769-53b4-4b49-ad3f-4b953b09539f admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:18.000398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe4769-53b4-4b49-ad3f-4b953b09539f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:18.002409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe4769-53b4-4b49-ad3f-4b953b09539f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:18.092283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 990/2010] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:17 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:18.093377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298c718-652b-412a-a919-0d7b01dc7395 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:18.104758 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298c718-652b-412a-a919-0d7b01dc7395 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:18.163421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be181389-b0af-4825-8dfd-054c4aacad4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:18.164099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be181389-b0af-4825-8dfd-054c4aacad4e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:18.164384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be181389-b0af-4825-8dfd-054c4aacad4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:18.164682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be181389-b0af-4825-8dfd-054c4aacad4e admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:18.165348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be181389-b0af-4825-8dfd-054c4aacad4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:18.167386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be181389-b0af-4825-8dfd-054c4aacad4e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:18.272625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1021/2011] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:18 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:18.276684 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe4769-53b4-4b49-ad3f-4b953b09539f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:18.287570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe4769-53b4-4b49-ad3f-4b953b09539f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:18.292194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a6e67-5cf2-4c09-ab25-05a2b2d3637e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:18.292840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a6e67-5cf2-4c09-ab25-05a2b2d3637e admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:18.293170 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a6e67-5cf2-4c09-ab25-05a2b2d3637e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:18.293503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a6e67-5cf2-4c09-ab25-05a2b2d3637e admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:18.298691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a6e67-5cf2-4c09-ab25-05a2b2d3637e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:18.298691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a6e67-5cf2-4c09-ab25-05a2b2d3637e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:18.307533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 991/2012] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:18.312080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be181389-b0af-4825-8dfd-054c4aacad4e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:18.323814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be181389-b0af-4825-8dfd-054c4aacad4e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:18.328970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9fdb1-f6a4-478c-93fc-ab1162a6624d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:18.329851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9fdb1-f6a4-478c-93fc-ab1162a6624d admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:18.330179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9fdb1-f6a4-478c-93fc-ab1162a6624d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:18.330462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9fdb1-f6a4-478c-93fc-ab1162a6624d admin admin] PATH_INFO: `/v3/projects/91d94e4e2cbb40458d592d7b77a84bfc/users/be62a73925e248ff84f2745d17dab95e/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:18.352908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9fdb1-f6a4-478c-93fc-ab1162a6624d admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d94e4e2cbb40458d592d7b77a84bfc, user_id=be62a73925e248ff84f2745d17dab95e, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:18.355684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9fdb1-f6a4-478c-93fc-ab1162a6624d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:18.453943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1022/2013] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:18 2022] PUT /identity/v3/projects/91d94e4e2cbb40458d592d7b77a84bfc/users/be62a73925e248ff84f2745d17dab95e/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:18.458066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a6e67-5cf2-4c09-ab25-05a2b2d3637e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:18.468235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a6e67-5cf2-4c09-ab25-05a2b2d3637e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:18.526738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa92369-a908-452d-b12b-104e67f0f989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:18.527415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa92369-a908-452d-b12b-104e67f0f989 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:18.527704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa92369-a908-452d-b12b-104e67f0f989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:18.531470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa92369-a908-452d-b12b-104e67f0f989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:18.531470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa92369-a908-452d-b12b-104e67f0f989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:18.531470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa92369-a908-452d-b12b-104e67f0f989 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:18.540613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 992/2014] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:18.544742 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9fdb1-f6a4-478c-93fc-ab1162a6624d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:18.555638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9fdb1-f6a4-478c-93fc-ab1162a6624d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:18.560319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc8e13-a2f7-4115-800c-128caaee21db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:18.561201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc8e13-a2f7-4115-800c-128caaee21db admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:18.561490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc8e13-a2f7-4115-800c-128caaee21db admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:18.561767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc8e13-a2f7-4115-800c-128caaee21db admin admin] PATH_INFO: `/v3/projects/91d94e4e2cbb40458d592d7b77a84bfc/users/be62a73925e248ff84f2745d17dab95e/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:18.563754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc8e13-a2f7-4115-800c-128caaee21db admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d94e4e2cbb40458d592d7b77a84bfc, user_id=be62a73925e248ff84f2745d17dab95e, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:18.567397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc8e13-a2f7-4115-800c-128caaee21db admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:18.577662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1023/2015] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:18 2022] PUT /identity/v3/projects/91d94e4e2cbb40458d592d7b77a84bfc/users/be62a73925e248ff84f2745d17dab95e/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:18.588074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2dbd-bffd-4485-8ef2-93c1fa015000 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:18.588367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2dbd-bffd-4485-8ef2-93c1fa015000 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:18.588646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2dbd-bffd-4485-8ef2-93c1fa015000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:18.606599 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:18.616962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-584a2dbd-bffd-4485-8ef2-93c1fa015000 None None] MFA Rules not processed for user `be62a73925e248ff84f2745d17dab95e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:18.618179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-584a2dbd-bffd-4485-8ef2-93c1fa015000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:18.692441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 993/2016] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:18 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:18.703655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc8e13-a2f7-4115-800c-128caaee21db admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:18.734836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:18.761245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc8e13-a2f7-4115-800c-128caaee21db admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:18.764825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bb4c3-44fe-4bfa-aee4-a1cd609fb24f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:18.765779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bb4c3-44fe-4bfa-aee4-a1cd609fb24f service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:18.766372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bb4c3-44fe-4bfa-aee4-a1cd609fb24f service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:18.766646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bb4c3-44fe-4bfa-aee4-a1cd609fb24f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:18.776766 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bb4c3-44fe-4bfa-aee4-a1cd609fb24f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:18.777815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bb4c3-44fe-4bfa-aee4-a1cd609fb24f service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:18.791532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1024/2017] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:18 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:18.977317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2dbd-bffd-4485-8ef2-93c1fa015000 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:19.032866 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bb4c3-44fe-4bfa-aee4-a1cd609fb24f service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:19.034897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0b4bb4c3-44fe-4bfa-aee4-a1cd609fb24f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:19.045582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bb4c3-44fe-4bfa-aee4-a1cd609fb24f service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:19.045802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2dbd-bffd-4485-8ef2-93c1fa015000 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:19.049202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65d2db-f881-4824-bc28-bd061ffc881b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:19.050483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65d2db-f881-4824-bc28-bd061ffc881b service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:19.050816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65d2db-f881-4824-bc28-bd061ffc881b service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:19.051011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:19.053663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65d2db-f881-4824-bc28-bd061ffc881b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:19.053819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:19.053819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:19.053819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] PATH_INFO: `/v3/users/a8a8e30d822f4c1689c29934f00377b2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:19.053819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a8e30d822f4c1689c29934f00377b2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:19.055591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:19.109422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65d2db-f881-4824-bc28-bd061ffc881b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:19.110904 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65d2db-f881-4824-bc28-bd061ffc881b service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:19.120218 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1025/2018] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:07:19.291938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a8e30d822f4c1689c29934f00377b2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:19.300721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a8e30d822f4c1689c29934f00377b2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:19.313141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a8e30d822f4c1689c29934f00377b2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:19.409704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a8e30d822f4c1689c29934f00377b2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:19.409704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a8e30d822f4c1689c29934f00377b2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:19.409704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 994/2019] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:18 2022] DELETE /identity/v3/users/a8a8e30d822f4c1689c29934f00377b2 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:19.413885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea4fb-cae7-4d9c-89a9-5362ee191210 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:19.414281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea4fb-cae7-4d9c-89a9-5362ee191210 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:19.414625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ea4fb-cae7-4d9c-89a9-5362ee191210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:19.438168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4b9ea4fb-cae7-4d9c-89a9-5362ee191210 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:19.439410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4b9ea4fb-cae7-4d9c-89a9-5362ee191210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:19.504110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1026/2020] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:19.514118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:19.515840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:19.533554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:19.577084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a5e0f-77af-410c-a427-f70aad04ffb6 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:19.580573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30521764-9f22-4e29-8919-b6b227050b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:19.582077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30521764-9f22-4e29-8919-b6b227050b86 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:19.582586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30521764-9f22-4e29-8919-b6b227050b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:19.582706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30521764-9f22-4e29-8919-b6b227050b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:19.593909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30521764-9f22-4e29-8919-b6b227050b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:19.595353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30521764-9f22-4e29-8919-b6b227050b86 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:19.605155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 995/2021] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:20.068253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ea4fb-cae7-4d9c-89a9-5362ee191210 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:20.069641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4b9ea4fb-cae7-4d9c-89a9-5362ee191210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:20.082430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ea4fb-cae7-4d9c-89a9-5362ee191210 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:20.090677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:20.091554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:20.091821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:20.092159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] PATH_INFO: `/v3/projects/8bf5884aaeab4c3bbe6521e634553bcb` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:20.098444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf5884aaeab4c3bbe6521e634553bcb)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:20.100714 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:20.211890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf5884aaeab4c3bbe6521e634553bcb', 'request_id': 'req-a05bc14a-1d6c-4108-861a-c3b360a59407'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:20.217496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf5884aaeab4c3bbe6521e634553bcb', 'request_id': 'req-a05bc14a-1d6c-4108-861a-c3b360a59407'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:20.228735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf5884aaeab4c3bbe6521e634553bcb', 'request_id': 'req-a05bc14a-1d6c-4108-861a-c3b360a59407'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:20.231500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:20.305824 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] The token cache is being invalidate because project 8bf5884aaeab4c3bbe6521e634553bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:20.306134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:20.306742 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:20.308306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1027/2022] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:20 2022] DELETE /identity/v3/projects/8bf5884aaeab4c3bbe6521e634553bcb => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:20.312298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30521764-9f22-4e29-8919-b6b227050b86 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:20.324175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30521764-9f22-4e29-8919-b6b227050b86 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:20.379390 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:20.380270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:20.380548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:20.380817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] PATH_INFO: `/v3/users/5ab25b2ec50844539f4680ea00156869` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:20.382116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ab25b2ec50844539f4680ea00156869)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:20.384037 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:20.543230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ab25b2ec50844539f4680ea00156869'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:20.552270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ab25b2ec50844539f4680ea00156869'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:20.564564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ab25b2ec50844539f4680ea00156869'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:20.655234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ab25b2ec50844539f4680ea00156869'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:20.655898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7034ea95-954a-4a2e-998d-32b879230f47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ab25b2ec50844539f4680ea00156869'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:20.662606 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 996/2023] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:20 2022] DELETE /identity/v3/users/5ab25b2ec50844539f4680ea00156869 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:21.090928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:21.092696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:21.108355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bc14a-1d6c-4108-861a-c3b360a59407 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:21.183756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:21.184911 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:21.185194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:21.185476 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] PATH_INFO: `/v3/projects/e820363244934a6f8e4b133dc256936d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:21.186757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e820363244934a6f8e4b133dc256936d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:21.188713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:21.324788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e820363244934a6f8e4b133dc256936d', 'request_id': 'req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:21.330453 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e820363244934a6f8e4b133dc256936d', 'request_id': 'req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:21.337703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e820363244934a6f8e4b133dc256936d', 'request_id': 'req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:21.415367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] The token cache is being invalidate because project e820363244934a6f8e4b133dc256936d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:21.415595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:21.416259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cbbc98a0-9a56-4ae5-a8a7-8eb513d4e5c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:21.417875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1028/2024] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:21 2022] DELETE /identity/v3/projects/e820363244934a6f8e4b133dc256936d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:21.426466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bc96f-6654-48fa-8c5c-bbb918e76050 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:21.426753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bc96f-6654-48fa-8c5c-bbb918e76050 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:21.427043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bc96f-6654-48fa-8c5c-bbb918e76050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:21.464919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-709bc96f-6654-48fa-8c5c-bbb918e76050 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:21.466155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-709bc96f-6654-48fa-8c5c-bbb918e76050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:21.529467 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:21.566186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 997/2025] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:21.574090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe47f8-8d8f-490f-85cd-0e6bef163986 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:21.574384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe47f8-8d8f-490f-85cd-0e6bef163986 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:21.574659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe47f8-8d8f-490f-85cd-0e6bef163986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:21.594972 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-63fe47f8-8d8f-490f-85cd-0e6bef163986 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:21.596917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-63fe47f8-8d8f-490f-85cd-0e6bef163986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:21.607454 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1029/2026] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:21.611465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bc96f-6654-48fa-8c5c-bbb918e76050 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:21.626266 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bc96f-6654-48fa-8c5c-bbb918e76050 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:21.632569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec807aea-784c-4a4b-9456-1865a2e7e653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:21.633236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec807aea-784c-4a4b-9456-1865a2e7e653 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:21.633515 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec807aea-784c-4a4b-9456-1865a2e7e653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:21.633954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec807aea-784c-4a4b-9456-1865a2e7e653 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:21.634749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec807aea-784c-4a4b-9456-1865a2e7e653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:21.635058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec807aea-784c-4a4b-9456-1865a2e7e653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:21.636635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec807aea-784c-4a4b-9456-1865a2e7e653 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:21.650080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 998/2027] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:07:21.659397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe47f8-8d8f-490f-85cd-0e6bef163986 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:21.668472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe47f8-8d8f-490f-85cd-0e6bef163986 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:21.675167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090900f9-dc85-4693-b405-bbdf2d47b7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:21.676050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090900f9-dc85-4693-b405-bbdf2d47b7ed admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:21.676267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090900f9-dc85-4693-b405-bbdf2d47b7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:21.677489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090900f9-dc85-4693-b405-bbdf2d47b7ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:21.677489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090900f9-dc85-4693-b405-bbdf2d47b7ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:21.679346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090900f9-dc85-4693-b405-bbdf2d47b7ed admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:21.787790 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1030/2028] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:21 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:21.789555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec807aea-784c-4a4b-9456-1865a2e7e653 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:21.801181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec807aea-784c-4a4b-9456-1865a2e7e653 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:21.857435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0956f-ce9e-4f7a-a373-cd84503f7c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:21.858126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0956f-ce9e-4f7a-a373-cd84503f7c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:21.858405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0956f-ce9e-4f7a-a373-cd84503f7c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:21.858763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0956f-ce9e-4f7a-a373-cd84503f7c3f admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:21.859454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0956f-ce9e-4f7a-a373-cd84503f7c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:21.861341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0956f-ce9e-4f7a-a373-cd84503f7c3f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:21.975999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 999/2029] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:21 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:21.976714 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090900f9-dc85-4693-b405-bbdf2d47b7ed admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:21.988898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090900f9-dc85-4693-b405-bbdf2d47b7ed admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:21.992772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c9b12-aae0-42d9-9298-2caac15c3f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:21.993429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c9b12-aae0-42d9-9298-2caac15c3f9b admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:21.993711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c9b12-aae0-42d9-9298-2caac15c3f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:21.993986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c9b12-aae0-42d9-9298-2caac15c3f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:21.994756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c9b12-aae0-42d9-9298-2caac15c3f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:21.996350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c9b12-aae0-42d9-9298-2caac15c3f9b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:22.007129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1031/2030] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:22.011074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0956f-ce9e-4f7a-a373-cd84503f7c3f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:22.021595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0956f-ce9e-4f7a-a373-cd84503f7c3f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:22.026436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707190c-f636-4e71-a48b-66dea1698c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:22.027108 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707190c-f636-4e71-a48b-66dea1698c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:22.027559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707190c-f636-4e71-a48b-66dea1698c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:22.027838 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707190c-f636-4e71-a48b-66dea1698c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:22.028559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707190c-f636-4e71-a48b-66dea1698c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:22.030202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707190c-f636-4e71-a48b-66dea1698c8c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:22.042086 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1000/2031] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:22.054992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c9b12-aae0-42d9-9298-2caac15c3f9b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:22.057427 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c9b12-aae0-42d9-9298-2caac15c3f9b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:22.062802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a2c8-3352-4e88-9710-892785c69b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:22.063683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a2c8-3352-4e88-9710-892785c69b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:22.063971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a2c8-3352-4e88-9710-892785c69b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:22.064252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a2c8-3352-4e88-9710-892785c69b7c admin admin] PATH_INFO: `/v3/projects/b11e7ef755dd4b54a5dfe04db7249855/users/98613c1d81a7400e9dd21353e156e960/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:22.080520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4a2c8-3352-4e88-9710-892785c69b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11e7ef755dd4b54a5dfe04db7249855, user_id=98613c1d81a7400e9dd21353e156e960, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:22.083803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4a2c8-3352-4e88-9710-892785c69b7c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:22.153076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1032/2032] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:22 2022] PUT /identity/v3/projects/b11e7ef755dd4b54a5dfe04db7249855/users/98613c1d81a7400e9dd21353e156e960/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:22.166660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa358-f9eb-4637-8304-4291e37c1b88 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:22.166949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa358-f9eb-4637-8304-4291e37c1b88 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:22.167225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa358-f9eb-4637-8304-4291e37c1b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:22.197124 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-758fa358-f9eb-4637-8304-4291e37c1b88 None None] MFA Rules not processed for user `98613c1d81a7400e9dd21353e156e960`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:22.199717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-758fa358-f9eb-4637-8304-4291e37c1b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:22.279892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1001/2033] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:22.293175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a2c8-3352-4e88-9710-892785c69b7c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:22.294515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-eec4a2c8-3352-4e88-9710-892785c69b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:22.320025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:22.379109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a2c8-3352-4e88-9710-892785c69b7c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:22.383162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be5735-236b-4f0d-98f6-5a825d6e4eab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf67fa55f4d64b7ebeeb1490453661e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:22.394679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be5735-236b-4f0d-98f6-5a825d6e4eab service glance] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:22.394679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be5735-236b-4f0d-98f6-5a825d6e4eab service glance] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:22.394679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be5735-236b-4f0d-98f6-5a825d6e4eab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:22.394679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be5735-236b-4f0d-98f6-5a825d6e4eab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:22.395222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be5735-236b-4f0d-98f6-5a825d6e4eab service glance] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:22.404130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1033/2034] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:23.443727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa358-f9eb-4637-8304-4291e37c1b88 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:23.445168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-758fa358-f9eb-4637-8304-4291e37c1b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:23.490122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:23.517257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa358-f9eb-4637-8304-4291e37c1b88 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:23.522248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:23.523922 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:23.524298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:23.524669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] PATH_INFO: `/v3/users/51aef575529144b584e5dbf487641e0e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:23.526277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51aef575529144b584e5dbf487641e0e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:23.528373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:23.788914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51aef575529144b584e5dbf487641e0e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:23.796496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51aef575529144b584e5dbf487641e0e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:23.808913 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51aef575529144b584e5dbf487641e0e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:23.906169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51aef575529144b584e5dbf487641e0e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:23.906933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51aef575529144b584e5dbf487641e0e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:23.914590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1002/2035] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:23 2022] DELETE /identity/v3/users/51aef575529144b584e5dbf487641e0e => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:23.918998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac84e38-44ce-4138-bb54-e3b327803109 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:23.919471 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac84e38-44ce-4138-bb54-e3b327803109 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:23.919851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac84e38-44ce-4138-bb54-e3b327803109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:23.940670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7ac84e38-44ce-4138-bb54-e3b327803109 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:23.943653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7ac84e38-44ce-4138-bb54-e3b327803109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:24.053929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1034/2036] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:24.064903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:24.067896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:24.137936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36559335-c414-44cd-a6b3-43bb4c6fbd83 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:24.143781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe44ac-7563-4c00-90cb-449825e4047a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:24.144528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe44ac-7563-4c00-90cb-449825e4047a service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:24.144930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe44ac-7563-4c00-90cb-449825e4047a service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:24.145227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe44ac-7563-4c00-90cb-449825e4047a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:24.167389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe44ac-7563-4c00-90cb-449825e4047a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:24.168890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe44ac-7563-4c00-90cb-449825e4047a service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:24.180575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1003/2037] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:24.494921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac84e38-44ce-4138-bb54-e3b327803109 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:24.496682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7ac84e38-44ce-4138-bb54-e3b327803109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:24.510628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac84e38-44ce-4138-bb54-e3b327803109 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:24.515667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:24.516671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:24.516814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:24.517095 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] PATH_INFO: `/v3/users/6bc8fc0c9cd440ca826a0495b3940010` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:24.518376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc8fc0c9cd440ca826a0495b3940010)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:24.520322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:24.829071 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc8fc0c9cd440ca826a0495b3940010'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:24.836388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc8fc0c9cd440ca826a0495b3940010'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:24.863941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe44ac-7563-4c00-90cb-449825e4047a service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:24.865481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-aefe44ac-7563-4c00-90cb-449825e4047a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:24.883482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe44ac-7563-4c00-90cb-449825e4047a service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:24.889404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:24.890112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:24.890112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:24.890539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] PATH_INFO: `/v3/projects/903774042ada4832a68353809fe4b5d8` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:24.892135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=903774042ada4832a68353809fe4b5d8)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:24.895238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:24.939201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc8fc0c9cd440ca826a0495b3940010'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:24.939888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc8fc0c9cd440ca826a0495b3940010'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:24.940607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-aacb9be9-8b49-44ec-b2b4-7b5e43bd8f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc8fc0c9cd440ca826a0495b3940010'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:24.960027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1035/2038] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:24 2022] DELETE /identity/v3/users/6bc8fc0c9cd440ca826a0495b3940010 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:24.964341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658d007-a86e-4e50-a464-6dda864dfb88 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:24.964626 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658d007-a86e-4e50-a464-6dda864dfb88 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:24.964901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658d007-a86e-4e50-a464-6dda864dfb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:24.984014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4658d007-a86e-4e50-a464-6dda864dfb88 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:24.985215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4658d007-a86e-4e50-a464-6dda864dfb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:24.996269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:25.051445 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1036/2039] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:25.061448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:25.062025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658d007-a86e-4e50-a464-6dda864dfb88 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:25.063288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903774042ada4832a68353809fe4b5d8', 'request_id': 'req-adbcba98-0f30-4237-bbe6-e9d100597b6a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:25.064280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4658d007-a86e-4e50-a464-6dda864dfb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:25.071835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903774042ada4832a68353809fe4b5d8', 'request_id': 'req-adbcba98-0f30-4237-bbe6-e9d100597b6a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:25.132520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658d007-a86e-4e50-a464-6dda864dfb88 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:25.197243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02ea97-cbbe-4732-aa90-d51e06077bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:25.197960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02ea97-cbbe-4732-aa90-d51e06077bec service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:25.198331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02ea97-cbbe-4732-aa90-d51e06077bec service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:25.198673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02ea97-cbbe-4732-aa90-d51e06077bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:25.226804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903774042ada4832a68353809fe4b5d8', 'request_id': 'req-adbcba98-0f30-4237-bbe6-e9d100597b6a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:25.246800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] The token cache is being invalidate because project 903774042ada4832a68353809fe4b5d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:25.247591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:25.248510 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:25.250210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1004/2040] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:24 2022] DELETE /identity/v3/projects/903774042ada4832a68353809fe4b5d8 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:25.254353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:25.256678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:25.299861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02ea97-cbbe-4732-aa90-d51e06077bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:25.302383 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02ea97-cbbe-4732-aa90-d51e06077bec service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:25.315529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1037/2041] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:07:25.357445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcba98-0f30-4237-bbe6-e9d100597b6a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:25.368631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:25.370170 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:25.371273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:25.372073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] PATH_INFO: `/v3/users/32c692563d814b37a40dd1b64a480167` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:25.373628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c692563d814b37a40dd1b64a480167)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:25.376296 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:25.606351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c692563d814b37a40dd1b64a480167'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:25.614290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c692563d814b37a40dd1b64a480167'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:25.626729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c692563d814b37a40dd1b64a480167'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:25.717737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c692563d814b37a40dd1b64a480167'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:25.727036 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c692563d814b37a40dd1b64a480167'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:25.728376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1005/2042] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:25 2022] DELETE /identity/v3/users/32c692563d814b37a40dd1b64a480167 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:26.012259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02ea97-cbbe-4732-aa90-d51e06077bec service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:26.013897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0a02ea97-cbbe-4732-aa90-d51e06077bec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:26.029166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02ea97-cbbe-4732-aa90-d51e06077bec service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:26.052057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:26.100760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:26.101662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:26.101944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:26.102256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] PATH_INFO: `/v3/projects/aed4816595a14ad9a0037172bf2ecfcd` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:26.103460 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed4816595a14ad9a0037172bf2ecfcd)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:26.105565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:26.220037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed4816595a14ad9a0037172bf2ecfcd', 'request_id': 'req-73e72e54-90a1-4fb8-a151-38f2bb110f8a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.226760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed4816595a14ad9a0037172bf2ecfcd', 'request_id': 'req-73e72e54-90a1-4fb8-a151-38f2bb110f8a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.240542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed4816595a14ad9a0037172bf2ecfcd', 'request_id': 'req-73e72e54-90a1-4fb8-a151-38f2bb110f8a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.243826 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:26.246268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:26.260418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d54f47-87ef-4061-8be2-381d0f83e6f2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:26.328086 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:26.330325 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] The token cache is being invalidate because project aed4816595a14ad9a0037172bf2ecfcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:26.331039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.331825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.334435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1038/2043] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:26 2022] DELETE /identity/v3/projects/aed4816595a14ad9a0037172bf2ecfcd => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:26.337253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:26.338478 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:26.338613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:26.338842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:26.339122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] PATH_INFO: `/v3/projects/d3670d2a04aa477abcdf974cf9d2def0` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:26.340498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:26.340834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3670d2a04aa477abcdf974cf9d2def0)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:26.345148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:26.364074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e72e54-90a1-4fb8-a151-38f2bb110f8a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:26.371733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:26.372641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:26.373461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:26.373755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] PATH_INFO: `/v3/users/adea65d14bd748278f72a8c0e27fc282` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:26.375744 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=adea65d14bd748278f72a8c0e27fc282)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:26.380886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:26.496040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3670d2a04aa477abcdf974cf9d2def0', 'request_id': 'req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.502935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3670d2a04aa477abcdf974cf9d2def0', 'request_id': 'req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.574612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3670d2a04aa477abcdf974cf9d2def0', 'request_id': 'req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.586802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] The token cache is being invalidate because project d3670d2a04aa477abcdf974cf9d2def0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:26.587122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.587783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.589445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1006/2044] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:26 2022] DELETE /identity/v3/projects/d3670d2a04aa477abcdf974cf9d2def0 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:26.593008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:26.594394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:26.595008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adea65d14bd748278f72a8c0e27fc282'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.623861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adea65d14bd748278f72a8c0e27fc282'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.674901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41085e-4cb4-4f8c-a0e6-a1c2c6ba61f8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:26.680803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:26.681816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:26.682134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:26.682418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] PATH_INFO: `/v3/users/992d109e1554474b9a8684fb6265da01` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:26.683691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] RBAC: Authorizing `identity:delete_user(user_id=992d109e1554474b9a8684fb6265da01)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:26.685610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:26.711534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adea65d14bd748278f72a8c0e27fc282'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.712242 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adea65d14bd748278f72a8c0e27fc282'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.712971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adea65d14bd748278f72a8c0e27fc282'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.734058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1039/2045] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:26 2022] DELETE /identity/v3/users/adea65d14bd748278f72a8c0e27fc282 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:26.734058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:26.734338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:26.764415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:26.802970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0719cf0-1de3-4d78-868c-73b976e759ea admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:26.807901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:26.808773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:26.809048 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:26.809321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] PATH_INFO: `/v3/users/fbebfe75bcc2418eb9f1f82143b71ff4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:26.810577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbebfe75bcc2418eb9f1f82143b71ff4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:26.812578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:26.913082 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992d109e1554474b9a8684fb6265da01'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.919702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992d109e1554474b9a8684fb6265da01'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:26.932746 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992d109e1554474b9a8684fb6265da01'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.042189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992d109e1554474b9a8684fb6265da01'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.042778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4fb17644-6b23-47f4-8d22-479948cf475e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992d109e1554474b9a8684fb6265da01'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.049312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1007/2046] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:26 2022] DELETE /identity/v3/users/992d109e1554474b9a8684fb6265da01 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:27.053469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dab2df-17c2-45b9-945e-4a98542422bd None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:27.053755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dab2df-17c2-45b9-945e-4a98542422bd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:27.054039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dab2df-17c2-45b9-945e-4a98542422bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:27.057278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbebfe75bcc2418eb9f1f82143b71ff4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.066230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbebfe75bcc2418eb9f1f82143b71ff4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.072635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:27.074620 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c0dab2df-17c2-45b9-945e-4a98542422bd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:27.076025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c0dab2df-17c2-45b9-945e-4a98542422bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:27.138624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1008/2047] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:27.155273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dab2df-17c2-45b9-945e-4a98542422bd None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:27.156699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c0dab2df-17c2-45b9-945e-4a98542422bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:27.195910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbebfe75bcc2418eb9f1f82143b71ff4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.196675 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbebfe75bcc2418eb9f1f82143b71ff4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.197296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbebfe75bcc2418eb9f1f82143b71ff4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.217131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1040/2048] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:26 2022] DELETE /identity/v3/users/fbebfe75bcc2418eb9f1f82143b71ff4 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:27.219400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dab2df-17c2-45b9-945e-4a98542422bd None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:27.277277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35e416-9018-4034-997c-59e891b99b03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:27.278005 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e416-9018-4034-997c-59e891b99b03 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:27.278317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e416-9018-4034-997c-59e891b99b03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:27.278594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e416-9018-4034-997c-59e891b99b03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:27.280508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1f35e416-9018-4034-997c-59e891b99b03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:27.353939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35e416-9018-4034-997c-59e891b99b03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:27.355541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35e416-9018-4034-997c-59e891b99b03 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:27.366560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1009/2049] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:07:27.618392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:27.619866 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:27.637027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0ef59-1658-42cd-9531-bae629916fb6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:27.644394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:27.645268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:27.645551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:27.645878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] PATH_INFO: `/v3/projects/9fd2cd0b17014208ad0bab1b0ecd91d6` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:27.647257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fd2cd0b17014208ad0bab1b0ecd91d6)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:27.649189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:27.771938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35e416-9018-4034-997c-59e891b99b03 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:27.773925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1f35e416-9018-4034-997c-59e891b99b03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:27.787102 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35e416-9018-4034-997c-59e891b99b03 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:27.798049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fd2cd0b17014208ad0bab1b0ecd91d6', 'request_id': 'req-06bb1243-2a28-48cc-9168-9d0ecdfbf474'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.805204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fd2cd0b17014208ad0bab1b0ecd91d6', 'request_id': 'req-06bb1243-2a28-48cc-9168-9d0ecdfbf474'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.815482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fd2cd0b17014208ad0bab1b0ecd91d6', 'request_id': 'req-06bb1243-2a28-48cc-9168-9d0ecdfbf474'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.870659 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:27.871672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:27.872020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:27.872367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] PATH_INFO: `/v3/projects/5042e05651f84f73b4a2eb63a066f72b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:27.873691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5042e05651f84f73b4a2eb63a066f72b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:27.876113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:27.883321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] The token cache is being invalidate because project 9fd2cd0b17014208ad0bab1b0ecd91d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:27.885039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.885039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1243-2a28-48cc-9168-9d0ecdfbf474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:27.886120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1041/2050] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:27 2022] DELETE /identity/v3/projects/9fd2cd0b17014208ad0bab1b0ecd91d6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:27.891397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3c0ca-f628-41fc-8054-2a59cf95d02f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:27.892019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3c0ca-f628-41fc-8054-2a59cf95d02f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:27.892485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3c0ca-f628-41fc-8054-2a59cf95d02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:27.914995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-56f3c0ca-f628-41fc-8054-2a59cf95d02f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:27.917092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-56f3c0ca-f628-41fc-8054-2a59cf95d02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:27.928958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1042/2051] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:27.939618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc7ca8-04f9-4d3f-9356-c088c65d73b9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:27.940308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc7ca8-04f9-4d3f-9356-c088c65d73b9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:27.940666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc7ca8-04f9-4d3f-9356-c088c65d73b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:27.972828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f4fc7ca8-04f9-4d3f-9356-c088c65d73b9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:27.974983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f4fc7ca8-04f9-4d3f-9356-c088c65d73b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:27.997492 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:27.999192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5042e05651f84f73b4a2eb63a066f72b', 'request_id': 'req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.006549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5042e05651f84f73b4a2eb63a066f72b', 'request_id': 'req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.055411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1043/2052] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:28.060495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5042e05651f84f73b4a2eb63a066f72b', 'request_id': 'req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.061986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc7ca8-04f9-4d3f-9356-c088c65d73b9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.062949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:28.074553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc7ca8-04f9-4d3f-9356-c088c65d73b9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.081123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2842fe8-1a78-45d8-b68c-0ca27ca920ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.082194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2842fe8-1a78-45d8-b68c-0ca27ca920ec admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.082678 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2842fe8-1a78-45d8-b68c-0ca27ca920ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.082797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] The token cache is being invalidate because project 5042e05651f84f73b4a2eb63a066f72b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:28.083040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.083550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2842fe8-1a78-45d8-b68c-0ca27ca920ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.084344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0cb63a4b-4b55-4e10-a7e4-c1b805c46e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.084731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2842fe8-1a78-45d8-b68c-0ca27ca920ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:28.085246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2842fe8-1a78-45d8-b68c-0ca27ca920ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.086361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1010/2053] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:27 2022] DELETE /identity/v3/projects/5042e05651f84f73b4a2eb63a066f72b => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:28.087916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2842fe8-1a78-45d8-b68c-0ca27ca920ec admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.091341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0b23f-e6ac-43d9-bfde-6b29e07354ef None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.091471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0b23f-e6ac-43d9-bfde-6b29e07354ef None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.091682 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0b23f-e6ac-43d9-bfde-6b29e07354ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.105289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1044/2054] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:07:28.109661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2842fe8-1a78-45d8-b68c-0ca27ca920ec admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.111816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e2842fe8-1a78-45d8-b68c-0ca27ca920ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:28.122402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-89b0b23f-e6ac-43d9-bfde-6b29e07354ef None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:28.126493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-89b0b23f-e6ac-43d9-bfde-6b29e07354ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:28.126667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2842fe8-1a78-45d8-b68c-0ca27ca920ec admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.131428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2b365-5644-467d-a5b7-5c4744f2cd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.132087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2b365-5644-467d-a5b7-5c4744f2cd39 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.132398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1011/2055] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:28.134240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2b365-5644-467d-a5b7-5c4744f2cd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.134557 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2b365-5644-467d-a5b7-5c4744f2cd39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.135277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2b365-5644-467d-a5b7-5c4744f2cd39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.137310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2b365-5644-467d-a5b7-5c4744f2cd39 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.140834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d994c-747c-4e1a-b267-62a09c80f2a2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.141121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d994c-747c-4e1a-b267-62a09c80f2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.141394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d994c-747c-4e1a-b267-62a09c80f2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.163942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-247d994c-747c-4e1a-b267-62a09c80f2a2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:28.165174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-247d994c-747c-4e1a-b267-62a09c80f2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:28.173278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1012/2056] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:28.177416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d994c-747c-4e1a-b267-62a09c80f2a2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.186383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d994c-747c-4e1a-b267-62a09c80f2a2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.191148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6776a4-f4be-4a38-bfea-9c4ce7a4d55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.191786 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6776a4-f4be-4a38-bfea-9c4ce7a4d55d admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.192072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6776a4-f4be-4a38-bfea-9c4ce7a4d55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.192350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6776a4-f4be-4a38-bfea-9c4ce7a4d55d admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.193100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6776a4-f4be-4a38-bfea-9c4ce7a4d55d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:28.193413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6776a4-f4be-4a38-bfea-9c4ce7a4d55d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.195061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6776a4-f4be-4a38-bfea-9c4ce7a4d55d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.208352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1013/2057] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:07:28.210785 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6776a4-f4be-4a38-bfea-9c4ce7a4d55d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.212609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0b6776a4-f4be-4a38-bfea-9c4ce7a4d55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:28.227471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6776a4-f4be-4a38-bfea-9c4ce7a4d55d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.232776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.233677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.233964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.234272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] PATH_INFO: `/v3/projects/3898a3ddbb114254b3cd132bfe55ffe0` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.235567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3898a3ddbb114254b3cd132bfe55ffe0)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.237613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.246695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1045/2058] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:28 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:28.246695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2b365-5644-467d-a5b7-5c4744f2cd39 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.308268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2b365-5644-467d-a5b7-5c4744f2cd39 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.312953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095459be-65b2-4b90-9c3b-7b9c38c02dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.313607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095459be-65b2-4b90-9c3b-7b9c38c02dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.313888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095459be-65b2-4b90-9c3b-7b9c38c02dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.314194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095459be-65b2-4b90-9c3b-7b9c38c02dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.315076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095459be-65b2-4b90-9c3b-7b9c38c02dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.317055 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095459be-65b2-4b90-9c3b-7b9c38c02dcf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.360719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3898a3ddbb114254b3cd132bfe55ffe0', 'request_id': 'req-875f66da-feb2-4883-99b9-66678537072a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.366521 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3898a3ddbb114254b3cd132bfe55ffe0', 'request_id': 'req-875f66da-feb2-4883-99b9-66678537072a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.456984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3898a3ddbb114254b3cd132bfe55ffe0', 'request_id': 'req-875f66da-feb2-4883-99b9-66678537072a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.466399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1046/2059] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:28 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:28.467335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:28.467676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095459be-65b2-4b90-9c3b-7b9c38c02dcf admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.471888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] The token cache is being invalidate because project 3898a3ddbb114254b3cd132bfe55ffe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:28.472173 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.472939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.474654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1014/2060] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:28 2022] DELETE /identity/v3/projects/3898a3ddbb114254b3cd132bfe55ffe0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:28.475389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.478483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:28.535634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095459be-65b2-4b90-9c3b-7b9c38c02dcf admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.540421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bd0c7-2f0c-4517-b017-1752562d7957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.541192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bd0c7-2f0c-4517-b017-1752562d7957 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.541572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bd0c7-2f0c-4517-b017-1752562d7957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.541854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bd0c7-2f0c-4517-b017-1752562d7957 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.542735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bd0c7-2f0c-4517-b017-1752562d7957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.544667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bd0c7-2f0c-4517-b017-1752562d7957 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.544832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f66da-feb2-4883-99b9-66678537072a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.549523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4cddc-ddbf-4a7d-a607-02a37efced62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.550212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4cddc-ddbf-4a7d-a607-02a37efced62 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.550532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4cddc-ddbf-4a7d-a607-02a37efced62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.550881 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4cddc-ddbf-4a7d-a607-02a37efced62 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.551613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4cddc-ddbf-4a7d-a607-02a37efced62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.553608 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4cddc-ddbf-4a7d-a607-02a37efced62 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.636460 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1047/2061] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:28 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:28.637395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bd0c7-2f0c-4517-b017-1752562d7957 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.640656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-945bd0c7-2f0c-4517-b017-1752562d7957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:28.654432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bd0c7-2f0c-4517-b017-1752562d7957 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.660116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.662061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.662061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.662301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1015/2062] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:28 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:28.662389 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] PATH_INFO: `/v3/users/be62a73925e248ff84f2745d17dab95e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.662463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4cddc-ddbf-4a7d-a607-02a37efced62 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.663390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] RBAC: Authorizing `identity:delete_user(user_id=be62a73925e248ff84f2745d17dab95e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.664145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a2d4cddc-ddbf-4a7d-a607-02a37efced62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:28.665691 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.676250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4cddc-ddbf-4a7d-a607-02a37efced62 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.683161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c1041-2b46-4495-b8a6-47afb6efc0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.684014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c1041-2b46-4495-b8a6-47afb6efc0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.684302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c1041-2b46-4495-b8a6-47afb6efc0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.684594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c1041-2b46-4495-b8a6-47afb6efc0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.685306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c1041-2b46-4495-b8a6-47afb6efc0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.686958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c1041-2b46-4495-b8a6-47afb6efc0b0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.699345 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1016/2063] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:28.700158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c1041-2b46-4495-b8a6-47afb6efc0b0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.710639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c1041-2b46-4495-b8a6-47afb6efc0b0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.715261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395ffac-65ef-4166-9013-f602ea03643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.715902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395ffac-65ef-4166-9013-f602ea03643f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.716183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395ffac-65ef-4166-9013-f602ea03643f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.716458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395ffac-65ef-4166-9013-f602ea03643f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.717174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395ffac-65ef-4166-9013-f602ea03643f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.718801 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395ffac-65ef-4166-9013-f602ea03643f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.732124 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1017/2064] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:28.732124 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395ffac-65ef-4166-9013-f602ea03643f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.739356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395ffac-65ef-4166-9013-f602ea03643f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.743985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7076a5-7111-4ce1-b309-eb39da3fd54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.744798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7076a5-7111-4ce1-b309-eb39da3fd54f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.744950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7076a5-7111-4ce1-b309-eb39da3fd54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.745169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7076a5-7111-4ce1-b309-eb39da3fd54f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.745874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7076a5-7111-4ce1-b309-eb39da3fd54f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.747459 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7076a5-7111-4ce1-b309-eb39da3fd54f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.759426 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1018/2065] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:28.759426 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7076a5-7111-4ce1-b309-eb39da3fd54f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.771489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7076a5-7111-4ce1-b309-eb39da3fd54f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.776200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64062f1e-fa1e-487f-afa8-ce7842256170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.776824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64062f1e-fa1e-487f-afa8-ce7842256170 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.777105 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64062f1e-fa1e-487f-afa8-ce7842256170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.777377 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64062f1e-fa1e-487f-afa8-ce7842256170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.778118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64062f1e-fa1e-487f-afa8-ce7842256170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.779768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64062f1e-fa1e-487f-afa8-ce7842256170 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.789389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1019/2066] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:28.790874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64062f1e-fa1e-487f-afa8-ce7842256170 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.801206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64062f1e-fa1e-487f-afa8-ce7842256170 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.807378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ff3c4-e874-4808-b01b-efbe1be2c100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.807536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ff3c4-e874-4808-b01b-efbe1be2c100 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.807689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ff3c4-e874-4808-b01b-efbe1be2c100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.807966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ff3c4-e874-4808-b01b-efbe1be2c100 admin admin] PATH_INFO: `/v3/projects/d5a484cba13e49698f12fe9f0fc8aab8/users/ac07db4664c348f08c85be2a6ae8d210/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.823208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ff3c4-e874-4808-b01b-efbe1be2c100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a484cba13e49698f12fe9f0fc8aab8, user_id=ac07db4664c348f08c85be2a6ae8d210, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:28.825804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ff3c4-e874-4808-b01b-efbe1be2c100 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:28.886390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be62a73925e248ff84f2745d17dab95e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.893644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be62a73925e248ff84f2745d17dab95e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.900633 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1020/2067] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:28 2022] PUT /identity/v3/projects/d5a484cba13e49698f12fe9f0fc8aab8/users/ac07db4664c348f08c85be2a6ae8d210/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:28.901564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ff3c4-e874-4808-b01b-efbe1be2c100 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:28.928858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:28.945575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be62a73925e248ff84f2745d17dab95e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.946230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be62a73925e248ff84f2745d17dab95e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.946847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-1ce1076c-c425-4051-b1cb-47b57d1a917c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be62a73925e248ff84f2745d17dab95e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:28.967856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ff3c4-e874-4808-b01b-efbe1be2c100 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:28.973602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1048/2068] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:28 2022] DELETE /identity/v3/users/be62a73925e248ff84f2745d17dab95e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:28.973602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc872c8c-e903-468b-b882-828d581001e6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.973602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc872c8c-e903-468b-b882-828d581001e6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.973602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc872c8c-e903-468b-b882-828d581001e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:28.980924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544d3ba-063b-4d06-abf5-49920fcc21f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:28.982311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544d3ba-063b-4d06-abf5-49920fcc21f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:28.982486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544d3ba-063b-4d06-abf5-49920fcc21f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:28.986411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544d3ba-063b-4d06-abf5-49920fcc21f3 admin admin] PATH_INFO: `/v3/projects/621ab3540def48f2ac9ab581ef09d557/users/bb72de6f83f24709987b2a3868975af8/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.004687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-dc872c8c-e903-468b-b882-828d581001e6 None None] MFA Rules not processed for user `ac07db4664c348f08c85be2a6ae8d210`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:29.005907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dc872c8c-e903-468b-b882-828d581001e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:29.006821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544d3ba-063b-4d06-abf5-49920fcc21f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=621ab3540def48f2ac9ab581ef09d557, user_id=bb72de6f83f24709987b2a3868975af8, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:29.010232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544d3ba-063b-4d06-abf5-49920fcc21f3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:29.086694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1049/2069] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:29.090537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc872c8c-e903-468b-b882-828d581001e6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:29.093030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dc872c8c-e903-468b-b882-828d581001e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:29.114008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1021/2070] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:28 2022] PUT /identity/v3/projects/621ab3540def48f2ac9ab581ef09d557/users/bb72de6f83f24709987b2a3868975af8/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:29.136231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a2350-6725-4080-8dbf-07b6b6406817 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:29.136231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a2350-6725-4080-8dbf-07b6b6406817 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:29.136231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a2350-6725-4080-8dbf-07b6b6406817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.145953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:29.169636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-114a2350-6725-4080-8dbf-07b6b6406817 None None] MFA Rules not processed for user `bb72de6f83f24709987b2a3868975af8`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:29.171017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-114a2350-6725-4080-8dbf-07b6b6406817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:29.181391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc872c8c-e903-468b-b882-828d581001e6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:29.188289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:29.282447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1022/2071] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:29.288918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a2350-6725-4080-8dbf-07b6b6406817 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:29.292013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-114a2350-6725-4080-8dbf-07b6b6406817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:29.311002 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345774be-45e5-448c-9d51-2078c6d548b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:29.312034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345774be-45e5-448c-9d51-2078c6d548b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:29.312646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345774be-45e5-448c-9d51-2078c6d548b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:29.313120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345774be-45e5-448c-9d51-2078c6d548b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.314214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345774be-45e5-448c-9d51-2078c6d548b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:29.317716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345774be-45e5-448c-9d51-2078c6d548b8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:29.401485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a2350-6725-4080-8dbf-07b6b6406817 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:29.411294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fe938-474d-440e-b238-2ec0c4748020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:29.411981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fe938-474d-440e-b238-2ec0c4748020 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:29.412456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fe938-474d-440e-b238-2ec0c4748020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:29.412897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fe938-474d-440e-b238-2ec0c4748020 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.414298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fe938-474d-440e-b238-2ec0c4748020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:29.416445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fe938-474d-440e-b238-2ec0c4748020 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:29.443226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1050/2072] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:29 2022] POST /identity/v3/projects => generated 383 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:29.444230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345774be-45e5-448c-9d51-2078c6d548b8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:29.514318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1023/2073] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:29 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:29.519523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fe938-474d-440e-b238-2ec0c4748020 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:29.542908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345774be-45e5-448c-9d51-2078c6d548b8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:29.609565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fe938-474d-440e-b238-2ec0c4748020 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:29.614893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb7c5e-49eb-467b-abc1-fbb6b339ebd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:29.616286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb7c5e-49eb-467b-abc1-fbb6b339ebd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:29.616286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb7c5e-49eb-467b-abc1-fbb6b339ebd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:29.616587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb7c5e-49eb-467b-abc1-fbb6b339ebd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.617296 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb7c5e-49eb-467b-abc1-fbb6b339ebd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:29.619527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb7c5e-49eb-467b-abc1-fbb6b339ebd4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:29.626836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aee77e-f694-4aa2-b732-d75a756d1aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:29.627690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aee77e-f694-4aa2-b732-d75a756d1aab admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:29.628174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aee77e-f694-4aa2-b732-d75a756d1aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:29.628570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aee77e-f694-4aa2-b732-d75a756d1aab admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.629367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aee77e-f694-4aa2-b732-d75a756d1aab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:29.634190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aee77e-f694-4aa2-b732-d75a756d1aab admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:29.717017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1024/2074] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:29 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:29.721721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb7c5e-49eb-467b-abc1-fbb6b339ebd4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:29.721721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-06cb7c5e-49eb-467b-abc1-fbb6b339ebd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:29.731680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1051/2075] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:29 2022] POST /identity/v3/users => generated 298 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:29.733107 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aee77e-f694-4aa2-b732-d75a756d1aab admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:29.737124 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb7c5e-49eb-467b-abc1-fbb6b339ebd4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:29.745279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:29.748494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:29.748633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aee77e-f694-4aa2-b732-d75a756d1aab admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:29.748757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:29.748757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] PATH_INFO: `/v3/projects/91d94e4e2cbb40458d592d7b77a84bfc` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.750399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d94e4e2cbb40458d592d7b77a84bfc)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:29.753170 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:29.755110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644021bd-e192-4615-9687-fd82ee21f959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:29.755799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644021bd-e192-4615-9687-fd82ee21f959 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:29.756087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644021bd-e192-4615-9687-fd82ee21f959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:29.756368 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644021bd-e192-4615-9687-fd82ee21f959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.757114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644021bd-e192-4615-9687-fd82ee21f959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:29.757484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:29.758778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644021bd-e192-4615-9687-fd82ee21f959 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:29.776641 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1052/2076] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:29.777775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644021bd-e192-4615-9687-fd82ee21f959 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:29.791241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644021bd-e192-4615-9687-fd82ee21f959 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:29.797434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7f7de-d744-464b-a4de-3190edf04d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:29.798273 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7f7de-d744-464b-a4de-3190edf04d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:29.798730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7f7de-d744-464b-a4de-3190edf04d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:29.799129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7f7de-d744-464b-a4de-3190edf04d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.799913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7f7de-d744-464b-a4de-3190edf04d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:29.801596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7f7de-d744-464b-a4de-3190edf04d02 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:29.816666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1053/2077] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:29.817593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7f7de-d744-464b-a4de-3190edf04d02 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:29.828498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7f7de-d744-464b-a4de-3190edf04d02 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:29.834545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e3136-6a83-4fa1-b181-ecb6c593b616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:29.835287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e3136-6a83-4fa1-b181-ecb6c593b616 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:29.835574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e3136-6a83-4fa1-b181-ecb6c593b616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:29.835853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e3136-6a83-4fa1-b181-ecb6c593b616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.836591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e3136-6a83-4fa1-b181-ecb6c593b616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:29.838292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e3136-6a83-4fa1-b181-ecb6c593b616 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:29.841143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:29.850538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1054/2078] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:29.851362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e3136-6a83-4fa1-b181-ecb6c593b616 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:29.866703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e3136-6a83-4fa1-b181-ecb6c593b616 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:29.874187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d94e4e2cbb40458d592d7b77a84bfc', 'request_id': 'req-8788e258-84f1-4852-9338-1db1cd8af4f3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:29.882869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d94e4e2cbb40458d592d7b77a84bfc', 'request_id': 'req-8788e258-84f1-4852-9338-1db1cd8af4f3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:29.932515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed15043-e11e-4f77-a382-f9cb392a20ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:29.933422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed15043-e11e-4f77-a382-f9cb392a20ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:29.933711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed15043-e11e-4f77-a382-f9cb392a20ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:29.933993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed15043-e11e-4f77-a382-f9cb392a20ea admin admin] PATH_INFO: `/v3/projects/039f4437d496490f86be87589f53e3c7/users/1c0c9a340a3c4fa694e97672628857e9/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:29.950672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed15043-e11e-4f77-a382-f9cb392a20ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=039f4437d496490f86be87589f53e3c7, user_id=1c0c9a340a3c4fa694e97672628857e9, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:29.953727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed15043-e11e-4f77-a382-f9cb392a20ea admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:29.974802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d94e4e2cbb40458d592d7b77a84bfc', 'request_id': 'req-8788e258-84f1-4852-9338-1db1cd8af4f3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:29.991144 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] The token cache is being invalidate because project 91d94e4e2cbb40458d592d7b77a84bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:29.991719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:29.992801 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:29.994582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1025/2079] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:29 2022] DELETE /identity/v3/projects/91d94e4e2cbb40458d592d7b77a84bfc => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:29.995839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:29.996792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:30.012483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788e258-84f1-4852-9338-1db1cd8af4f3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:30.017798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82beb3ba-472c-41e6-84da-cc1edd97d672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:30.019475 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beb3ba-472c-41e6-84da-cc1edd97d672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.020068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beb3ba-472c-41e6-84da-cc1edd97d672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.020318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beb3ba-472c-41e6-84da-cc1edd97d672 admin admin] PATH_INFO: `/v3/projects/97dcd1788b544599aa6d19e343c0b4cb/users/3da896b6fc764233a1fc331851026870/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.039281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82beb3ba-472c-41e6-84da-cc1edd97d672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97dcd1788b544599aa6d19e343c0b4cb, user_id=3da896b6fc764233a1fc331851026870, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:30.041970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82beb3ba-472c-41e6-84da-cc1edd97d672 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:30.049942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1055/2080] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:29 2022] PUT /identity/v3/projects/039f4437d496490f86be87589f53e3c7/users/1c0c9a340a3c4fa694e97672628857e9/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:30.051548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc90b35-ebfa-4ac4-a740-53724086335f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.051906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc90b35-ebfa-4ac4-a740-53724086335f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.052567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc90b35-ebfa-4ac4-a740-53724086335f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.070644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9dc90b35-ebfa-4ac4-a740-53724086335f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:30.071872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9dc90b35-ebfa-4ac4-a740-53724086335f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:30.140666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1056/2081] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:30.141469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc90b35-ebfa-4ac4-a740-53724086335f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:30.142913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9dc90b35-ebfa-4ac4-a740-53724086335f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:30.159308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc90b35-ebfa-4ac4-a740-53724086335f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:30.162308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1026/2082] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:29 2022] PUT /identity/v3/projects/97dcd1788b544599aa6d19e343c0b4cb/users/3da896b6fc764233a1fc331851026870/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:30.162749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1bfc-d941-4b93-8f12-5e5a13ced743 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.163040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1bfc-d941-4b93-8f12-5e5a13ced743 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.163377 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1bfc-d941-4b93-8f12-5e5a13ced743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.183846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3eab1bfc-d941-4b93-8f12-5e5a13ced743 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:30.191535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3eab1bfc-d941-4b93-8f12-5e5a13ced743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:30.232810 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5076a23-df1c-44d9-bb1e-a54ea700ce31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:30.233489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5076a23-df1c-44d9-bb1e-a54ea700ce31 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.233769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5076a23-df1c-44d9-bb1e-a54ea700ce31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.234065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5076a23-df1c-44d9-bb1e-a54ea700ce31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.234807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5076a23-df1c-44d9-bb1e-a54ea700ce31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:30.236555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5076a23-df1c-44d9-bb1e-a54ea700ce31 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:30.248128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1057/2083] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:30.249806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349bbb3-58b5-4dcf-ba9f-6fb07c06115f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.250139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349bbb3-58b5-4dcf-ba9f-6fb07c06115f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.250552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349bbb3-58b5-4dcf-ba9f-6fb07c06115f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.259288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1027/2084] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:30.260349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab1bfc-d941-4b93-8f12-5e5a13ced743 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:30.276006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab1bfc-d941-4b93-8f12-5e5a13ced743 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:30.280833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf30bd-95cf-4393-91e8-44ed9cb81ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:30.281704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf30bd-95cf-4393-91e8-44ed9cb81ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.281817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b349bbb3-58b5-4dcf-ba9f-6fb07c06115f None None] MFA Rules not processed for user `3da896b6fc764233a1fc331851026870`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:30.281993 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf30bd-95cf-4393-91e8-44ed9cb81ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.282307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf30bd-95cf-4393-91e8-44ed9cb81ae7 admin admin] PATH_INFO: `/v3/projects/039f4437d496490f86be87589f53e3c7/users/1c0c9a340a3c4fa694e97672628857e9/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.282978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b349bbb3-58b5-4dcf-ba9f-6fb07c06115f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:30.285089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf30bd-95cf-4393-91e8-44ed9cb81ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=039f4437d496490f86be87589f53e3c7, user_id=1c0c9a340a3c4fa694e97672628857e9, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:30.287909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf30bd-95cf-4393-91e8-44ed9cb81ae7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:30.289312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:30.302212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1028/2085] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:30 2022] PUT /identity/v3/projects/039f4437d496490f86be87589f53e3c7/users/1c0c9a340a3c4fa694e97672628857e9/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:30.304088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf30bd-95cf-4393-91e8-44ed9cb81ae7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:30.371388 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:30.381905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf30bd-95cf-4393-91e8-44ed9cb81ae7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:30.386580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c545d-3f54-444a-b7c2-3351b94a7b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:30.387240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c545d-3f54-444a-b7c2-3351b94a7b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.387523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c545d-3f54-444a-b7c2-3351b94a7b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.387791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c545d-3f54-444a-b7c2-3351b94a7b83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.388525 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c545d-3f54-444a-b7c2-3351b94a7b83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:30.388929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c545d-3f54-444a-b7c2-3351b94a7b83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:30.392224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c545d-3f54-444a-b7c2-3351b94a7b83 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:30.392352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1058/2086] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:30.393621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b55a2-2ed3-4637-8ce1-8ee29ca6a794 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.393950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b55a2-2ed3-4637-8ce1-8ee29ca6a794 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.394259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b55a2-2ed3-4637-8ce1-8ee29ca6a794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.407014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1029/2087] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:07:30.407014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c545d-3f54-444a-b7c2-3351b94a7b83 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:30.417963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c545d-3f54-444a-b7c2-3351b94a7b83 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:30.422910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fa5f0-7fef-4bbd-bbb1-7084531fcb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:30.423580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fa5f0-7fef-4bbd-bbb1-7084531fcb9d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.424721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fa5f0-7fef-4bbd-bbb1-7084531fcb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.425071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fa5f0-7fef-4bbd-bbb1-7084531fcb9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.426730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fa5f0-7fef-4bbd-bbb1-7084531fcb9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:30.437606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-132b55a2-2ed3-4637-8ce1-8ee29ca6a794 None None] MFA Rules not processed for user `1c0c9a340a3c4fa694e97672628857e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:30.437606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-132b55a2-2ed3-4637-8ce1-8ee29ca6a794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:30.437859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fa5f0-7fef-4bbd-bbb1-7084531fcb9d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:30.530022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1030/2088] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:30 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:30.531163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fa5f0-7fef-4bbd-bbb1-7084531fcb9d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:30.537102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1059/2089] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:30.538296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b55a2-2ed3-4637-8ce1-8ee29ca6a794 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:30.552856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b55a2-2ed3-4637-8ce1-8ee29ca6a794 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:30.658825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2538373-04a7-46dd-80c0-0c59dede0210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:30.659983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2538373-04a7-46dd-80c0-0c59dede0210 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.660265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2538373-04a7-46dd-80c0-0c59dede0210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.660539 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2538373-04a7-46dd-80c0-0c59dede0210 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.661238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2538373-04a7-46dd-80c0-0c59dede0210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:30.669779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2538373-04a7-46dd-80c0-0c59dede0210 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:30.677578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fa5f0-7fef-4bbd-bbb1-7084531fcb9d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:30.684577 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cfbcd-814d-48eb-8429-8916d1f58f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:30.685631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cfbcd-814d-48eb-8429-8916d1f58f1a admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.686221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cfbcd-814d-48eb-8429-8916d1f58f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.686705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cfbcd-814d-48eb-8429-8916d1f58f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.687650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823cfbcd-814d-48eb-8429-8916d1f58f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:30.691010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823cfbcd-814d-48eb-8429-8916d1f58f1a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:30.769356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1060/2090] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:30 2022] POST /identity/v3/users => generated 299 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:30.770415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2538373-04a7-46dd-80c0-0c59dede0210 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:30.775877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c2538373-04a7-46dd-80c0-0c59dede0210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:30.804527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:30.810066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1031/2091] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:30 2022] POST /identity/v3/projects => generated 369 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:30.810066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cfbcd-814d-48eb-8429-8916d1f58f1a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:30.840106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2538373-04a7-46dd-80c0-0c59dede0210 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:30.889334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cfbcd-814d-48eb-8429-8916d1f58f1a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:30.889334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccae67-a3cb-40de-ae43-1439bf5c5105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:30.890477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccae67-a3cb-40de-ae43-1439bf5c5105 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.890762 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccae67-a3cb-40de-ae43-1439bf5c5105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.891041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccae67-a3cb-40de-ae43-1439bf5c5105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.891777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccae67-a3cb-40de-ae43-1439bf5c5105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:30.893425 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccae67-a3cb-40de-ae43-1439bf5c5105 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:30.903674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a44ef-ccd1-491d-a8a8-c94ad777c159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:30.904352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1032/2092] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:30.905024 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a44ef-ccd1-491d-a8a8-c94ad777c159 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.905163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccae67-a3cb-40de-ae43-1439bf5c5105 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:30.905664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a44ef-ccd1-491d-a8a8-c94ad777c159 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.906053 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a44ef-ccd1-491d-a8a8-c94ad777c159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.920725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccae67-a3cb-40de-ae43-1439bf5c5105 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:30.926390 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aa524-93ca-4884-9296-be1e35012c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:30.928421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aa524-93ca-4884-9296-be1e35012c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:30.928780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aa524-93ca-4884-9296-be1e35012c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:30.929018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aa524-93ca-4884-9296-be1e35012c66 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:30.929733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521aa524-93ca-4884-9296-be1e35012c66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:30.931718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521aa524-93ca-4884-9296-be1e35012c66 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:30.985290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a44ef-ccd1-491d-a8a8-c94ad777c159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:30.987797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a44ef-ccd1-491d-a8a8-c94ad777c159 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:30.999086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1061/2093] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:31.001342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a44ef-ccd1-491d-a8a8-c94ad777c159 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:31.016561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a44ef-ccd1-491d-a8a8-c94ad777c159 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:31.021671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1033/2094] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:30 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:31.029391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71f162-059f-4cff-8499-e3003f49801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:31.029391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71f162-059f-4cff-8499-e3003f49801a admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.029391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71f162-059f-4cff-8499-e3003f49801a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.029391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71f162-059f-4cff-8499-e3003f49801a admin admin] PATH_INFO: `/v3/projects/cdc6f8bff7c54f2bb14f7cf2a7c538e6/users/7c658d5002b6452ba2d5c7b914d1a869/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.031286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aa524-93ca-4884-9296-be1e35012c66 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:31.047426 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aa524-93ca-4884-9296-be1e35012c66 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:31.049018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71f162-059f-4cff-8499-e3003f49801a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc6f8bff7c54f2bb14f7cf2a7c538e6, user_id=7c658d5002b6452ba2d5c7b914d1a869, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:31.053516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71f162-059f-4cff-8499-e3003f49801a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:31.062000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a8c0e-272a-417e-87eb-f3522918026f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:31.062000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a8c0e-272a-417e-87eb-f3522918026f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.062000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a8c0e-272a-417e-87eb-f3522918026f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.062000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a8c0e-272a-417e-87eb-f3522918026f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.062439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a8c0e-272a-417e-87eb-f3522918026f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:31.063059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a8c0e-272a-417e-87eb-f3522918026f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:31.078218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1034/2095] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:31.081908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a8c0e-272a-417e-87eb-f3522918026f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:31.093654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a8c0e-272a-417e-87eb-f3522918026f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:31.100050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e08fb4-b050-4034-ae7f-29b78a32f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:31.104273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e08fb4-b050-4034-ae7f-29b78a32f16a admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.104273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e08fb4-b050-4034-ae7f-29b78a32f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.104273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e08fb4-b050-4034-ae7f-29b78a32f16a admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.104273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e08fb4-b050-4034-ae7f-29b78a32f16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:31.104969 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e08fb4-b050-4034-ae7f-29b78a32f16a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:31.122070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1035/2096] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:31.123007 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e08fb4-b050-4034-ae7f-29b78a32f16a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:31.133641 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e08fb4-b050-4034-ae7f-29b78a32f16a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:31.138410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3112c27-d9f8-409b-b473-18e481aa4d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:31.139312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3112c27-d9f8-409b-b473-18e481aa4d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.139592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3112c27-d9f8-409b-b473-18e481aa4d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.139871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3112c27-d9f8-409b-b473-18e481aa4d48 admin admin] PATH_INFO: `/v3/projects/d766f73cac3f40bba6f961db2e005495/users/70b2afde553a455e841e440672584dac/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.143216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:31.160018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1062/2097] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:31 2022] PUT /identity/v3/projects/cdc6f8bff7c54f2bb14f7cf2a7c538e6/users/7c658d5002b6452ba2d5c7b914d1a869/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:31.163497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71f162-059f-4cff-8499-e3003f49801a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:31.164582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3112c27-d9f8-409b-b473-18e481aa4d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d766f73cac3f40bba6f961db2e005495, user_id=70b2afde553a455e841e440672584dac, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:31.167665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3112c27-d9f8-409b-b473-18e481aa4d48 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:31.235993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71f162-059f-4cff-8499-e3003f49801a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:31.241706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd81059-6c65-4260-bcaf-2ea09ba6fb3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:31.242489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd81059-6c65-4260-bcaf-2ea09ba6fb3d service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.242778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd81059-6c65-4260-bcaf-2ea09ba6fb3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.243059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd81059-6c65-4260-bcaf-2ea09ba6fb3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.244935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-abd81059-6c65-4260-bcaf-2ea09ba6fb3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:31.258964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1036/2098] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:31 2022] PUT /identity/v3/projects/d766f73cac3f40bba6f961db2e005495/users/70b2afde553a455e841e440672584dac/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:31.260114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3112c27-d9f8-409b-b473-18e481aa4d48 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:31.274985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:31.350365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd81059-6c65-4260-bcaf-2ea09ba6fb3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:31.355810 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd81059-6c65-4260-bcaf-2ea09ba6fb3d service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:31.355931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3112c27-d9f8-409b-b473-18e481aa4d48 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:31.361225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f4343-01a2-4f1e-bb4b-a2b8f71c1c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:31.362471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f4343-01a2-4f1e-bb4b-a2b8f71c1c6e admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.362757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f4343-01a2-4f1e-bb4b-a2b8f71c1c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.363034 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f4343-01a2-4f1e-bb4b-a2b8f71c1c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.363777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f4343-01a2-4f1e-bb4b-a2b8f71c1c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:31.365550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f4343-01a2-4f1e-bb4b-a2b8f71c1c6e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:31.375137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1063/2099] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:31.376617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddb8c-566a-441f-962d-64e23d4464ea None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.376908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddb8c-566a-441f-962d-64e23d4464ea None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.381351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddb8c-566a-441f-962d-64e23d4464ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.381887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1037/2100] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:31.384025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f4343-01a2-4f1e-bb4b-a2b8f71c1c6e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:31.395818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f4343-01a2-4f1e-bb4b-a2b8f71c1c6e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:31.402022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09595a-af06-455a-ac57-3e4246612b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:31.411044 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09595a-af06-455a-ac57-3e4246612b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.411044 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09595a-af06-455a-ac57-3e4246612b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.411044 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09595a-af06-455a-ac57-3e4246612b01 admin admin] PATH_INFO: `/v3/projects/cdc6f8bff7c54f2bb14f7cf2a7c538e6/users/7c658d5002b6452ba2d5c7b914d1a869/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.411044 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09595a-af06-455a-ac57-3e4246612b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc6f8bff7c54f2bb14f7cf2a7c538e6, user_id=7c658d5002b6452ba2d5c7b914d1a869, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:31.411044 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09595a-af06-455a-ac57-3e4246612b01 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:31.418336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-408ddb8c-566a-441f-962d-64e23d4464ea None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:31.419652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-408ddb8c-566a-441f-962d-64e23d4464ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:31.427349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1038/2101] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:31 2022] PUT /identity/v3/projects/cdc6f8bff7c54f2bb14f7cf2a7c538e6/users/7c658d5002b6452ba2d5c7b914d1a869/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:31.440375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc4361-3aac-4f78-87fc-945964769e1f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.441156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc4361-3aac-4f78-87fc-945964769e1f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.442374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc4361-3aac-4f78-87fc-945964769e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.482827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6edc4361-3aac-4f78-87fc-945964769e1f None None] MFA Rules not processed for user `7c658d5002b6452ba2d5c7b914d1a869`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:31.487254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6edc4361-3aac-4f78-87fc-945964769e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:31.554460 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1064/2102] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:31.561411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ddb8c-566a-441f-962d-64e23d4464ea None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:31.563202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-408ddb8c-566a-441f-962d-64e23d4464ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:31.597059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1039/2103] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:31.610445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc4361-3aac-4f78-87fc-945964769e1f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:31.612755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6edc4361-3aac-4f78-87fc-945964769e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:31.668104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ddb8c-566a-441f-962d-64e23d4464ea None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:31.674639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b6383-f753-4f15-aa0d-5e637d13594e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:31.675289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b6383-f753-4f15-aa0d-5e637d13594e service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.675686 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b6383-f753-4f15-aa0d-5e637d13594e service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.676067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b6383-f753-4f15-aa0d-5e637d13594e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.688494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b6383-f753-4f15-aa0d-5e637d13594e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:31.695157 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b6383-f753-4f15-aa0d-5e637d13594e service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:31.711753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1065/2104] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:07:31.725559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc4361-3aac-4f78-87fc-945964769e1f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:31.729688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505fcb3-27d4-4883-af4e-0334ab43ea5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:31.735705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505fcb3-27d4-4883-af4e-0334ab43ea5b service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:31.735705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505fcb3-27d4-4883-af4e-0334ab43ea5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:31.735705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505fcb3-27d4-4883-af4e-0334ab43ea5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:31.745843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3505fcb3-27d4-4883-af4e-0334ab43ea5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:31.754200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:31.844662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505fcb3-27d4-4883-af4e-0334ab43ea5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:31.847242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505fcb3-27d4-4883-af4e-0334ab43ea5b service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:31.861336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1040/2105] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:07:33.572381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37f5f6-cf63-43fc-9eaa-2c81e1067b9c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:33.573023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37f5f6-cf63-43fc-9eaa-2c81e1067b9c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:33.573023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37f5f6-cf63-43fc-9eaa-2c81e1067b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:33.603170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4f37f5f6-cf63-43fc-9eaa-2c81e1067b9c None None] MFA Rules not processed for user `70b2afde553a455e841e440672584dac`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:33.603877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4f37f5f6-cf63-43fc-9eaa-2c81e1067b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:33.672481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:33.720293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1066/2106] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:33.727753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505fcb3-27d4-4883-af4e-0334ab43ea5b service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:33.745793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505fcb3-27d4-4883-af4e-0334ab43ea5b service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:33.748070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ce35e-5121-4153-aa8b-5eebc51ec81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:33.748836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ce35e-5121-4153-aa8b-5eebc51ec81e admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:33.749173 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ce35e-5121-4153-aa8b-5eebc51ec81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:33.750921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ce35e-5121-4153-aa8b-5eebc51ec81e admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:33.751716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ce35e-5121-4153-aa8b-5eebc51ec81e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:33.754449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ce35e-5121-4153-aa8b-5eebc51ec81e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:33.847667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1041/2107] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:33 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:33.851506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37f5f6-cf63-43fc-9eaa-2c81e1067b9c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:33.865595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37f5f6-cf63-43fc-9eaa-2c81e1067b9c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:33.967993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a945a8-4a6f-4690-93bb-839fbbdcceed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:33.968792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a945a8-4a6f-4690-93bb-839fbbdcceed admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:33.969365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a945a8-4a6f-4690-93bb-839fbbdcceed admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:33.969834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a945a8-4a6f-4690-93bb-839fbbdcceed admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:33.972370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a945a8-4a6f-4690-93bb-839fbbdcceed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:33.974671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a945a8-4a6f-4690-93bb-839fbbdcceed admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:34.004886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6004177-cf32-49cf-b59f-28e1f48e33a3 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.005275 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6004177-cf32-49cf-b59f-28e1f48e33a3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.005604 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6004177-cf32-49cf-b59f-28e1f48e33a3 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.007975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1042/2108] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 09:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:07:34.013438 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5201a-fdaf-4599-adaa-7769aa39657d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.013800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5201a-fdaf-4599-adaa-7769aa39657d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.014102 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5201a-fdaf-4599-adaa-7769aa39657d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.036055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f5b5201a-fdaf-4599-adaa-7769aa39657d None None] MFA Rules not processed for user `bca7c1164d3e4833a10ffc37cbc6f092`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:34.037257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f5b5201a-fdaf-4599-adaa-7769aa39657d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:34.082270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1067/2109] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:33 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:34.082270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a945a8-4a6f-4690-93bb-839fbbdcceed admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:34.121287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1043/2110] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 09:07:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:34.134194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5201a-fdaf-4599-adaa-7769aa39657d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:34.163736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5201a-fdaf-4599-adaa-7769aa39657d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:34.163830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a945a8-4a6f-4690-93bb-839fbbdcceed admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:34.165770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b4e1a-d68f-463c-a532-db7afd4e360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:34.166575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b4e1a-d68f-463c-a532-db7afd4e360f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.167010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b4e1a-d68f-463c-a532-db7afd4e360f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.167357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b4e1a-d68f-463c-a532-db7afd4e360f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.168323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b4e1a-d68f-463c-a532-db7afd4e360f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:34.170165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b4e1a-d68f-463c-a532-db7afd4e360f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:34.171050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d6418-1bd8-4eba-9be2-6d9b96ab470a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:34.172937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d6418-1bd8-4eba-9be2-6d9b96ab470a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.173300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d6418-1bd8-4eba-9be2-6d9b96ab470a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.173805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d6418-1bd8-4eba-9be2-6d9b96ab470a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.185857 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1044/2111] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:34.186936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b4e1a-d68f-463c-a532-db7afd4e360f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:34.188728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0d4b4e1a-d68f-463c-a532-db7afd4e360f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:34.203892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b4e1a-d68f-463c-a532-db7afd4e360f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:34.212781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d26b7-c44e-4a51-bbfb-8b555e637466 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:34.213430 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d26b7-c44e-4a51-bbfb-8b555e637466 service placement] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.213709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d26b7-c44e-4a51-bbfb-8b555e637466 service placement] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.213979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d26b7-c44e-4a51-bbfb-8b555e637466 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.225428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d26b7-c44e-4a51-bbfb-8b555e637466 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:34.227049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d26b7-c44e-4a51-bbfb-8b555e637466 service placement] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:34.238120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1045/2112] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:34.238899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d26b7-c44e-4a51-bbfb-8b555e637466 service placement] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:34.240208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d31d26b7-c44e-4a51-bbfb-8b555e637466 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:34.244324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d6418-1bd8-4eba-9be2-6d9b96ab470a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:34.246016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d6418-1bd8-4eba-9be2-6d9b96ab470a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:34.255366 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d26b7-c44e-4a51-bbfb-8b555e637466 service placement] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:34.258620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1068/2113] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:34.259459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d6418-1bd8-4eba-9be2-6d9b96ab470a service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:34.260228 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:34.261416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.261798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.261983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] PATH_INFO: `/v3/users/9f6eb40fbef34931b668330b9f4db27d` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.263519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f6eb40fbef34931b668330b9f4db27d)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:34.266089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:34.285552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d6418-1bd8-4eba-9be2-6d9b96ab470a service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:34.298109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0a807-6565-42cc-a3d6-4a185b2bf5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:34.300229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a807-6565-42cc-a3d6-4a185b2bf5ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.300647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a807-6565-42cc-a3d6-4a185b2bf5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.310017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a807-6565-42cc-a3d6-4a185b2bf5ff admin admin] PATH_INFO: `/v3/projects/f19246c2607b475bbb1dc5abd1ad2567/users/f9dba100ffe24659b2b42e8617c3dc14/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.328702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0a807-6565-42cc-a3d6-4a185b2bf5ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19246c2607b475bbb1dc5abd1ad2567, user_id=f9dba100ffe24659b2b42e8617c3dc14, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:34.338697 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0a807-6565-42cc-a3d6-4a185b2bf5ff admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:34.450262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1069/2114] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:34 2022] PUT /identity/v3/projects/f19246c2607b475bbb1dc5abd1ad2567/users/f9dba100ffe24659b2b42e8617c3dc14/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:34.455014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0a807-6565-42cc-a3d6-4a185b2bf5ff admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:34.467663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:34.520299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0a807-6565-42cc-a3d6-4a185b2bf5ff admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:34.524975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e2454-8c23-4433-93b3-e8a7fe6b6b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:34.525629 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2454-8c23-4433-93b3-e8a7fe6b6b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.525929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2454-8c23-4433-93b3-e8a7fe6b6b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.526221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2454-8c23-4433-93b3-e8a7fe6b6b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.527036 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e2454-8c23-4433-93b3-e8a7fe6b6b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:34.528694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e2454-8c23-4433-93b3-e8a7fe6b6b91 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:34.543348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1070/2115] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:34.544253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e2454-8c23-4433-93b3-e8a7fe6b6b91 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:34.545722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f6eb40fbef34931b668330b9f4db27d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:34.547858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:34.554799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f6eb40fbef34931b668330b9f4db27d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:34.555044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e2454-8c23-4433-93b3-e8a7fe6b6b91 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:34.559767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dab97f-cc69-4560-96fa-5f22e8cca145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:34.560717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dab97f-cc69-4560-96fa-5f22e8cca145 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.561059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dab97f-cc69-4560-96fa-5f22e8cca145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.561333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dab97f-cc69-4560-96fa-5f22e8cca145 admin admin] PATH_INFO: `/v3/projects/f19246c2607b475bbb1dc5abd1ad2567/users/f9dba100ffe24659b2b42e8617c3dc14/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.564392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dab97f-cc69-4560-96fa-5f22e8cca145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19246c2607b475bbb1dc5abd1ad2567, user_id=f9dba100ffe24659b2b42e8617c3dc14, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:34.567816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dab97f-cc69-4560-96fa-5f22e8cca145 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:34.567945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f6eb40fbef34931b668330b9f4db27d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:34.577233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1071/2116] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:34 2022] PUT /identity/v3/projects/f19246c2607b475bbb1dc5abd1ad2567/users/f9dba100ffe24659b2b42e8617c3dc14/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:34.630530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f6eb40fbef34931b668330b9f4db27d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:34.631148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f6eb40fbef34931b668330b9f4db27d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:34.638141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1046/2117] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:34 2022] DELETE /identity/v3/users/9f6eb40fbef34931b668330b9f4db27d => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:34.773232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dab97f-cc69-4560-96fa-5f22e8cca145 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:34.775185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-85dab97f-cc69-4560-96fa-5f22e8cca145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:34.845194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dab97f-cc69-4560-96fa-5f22e8cca145 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:34.904507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:34.905386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.905664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.905931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] PATH_INFO: `/v3/users/ac07db4664c348f08c85be2a6ae8d210` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.907175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac07db4664c348f08c85be2a6ae8d210)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:34.909625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:34.928968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:34.931145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:34.947388 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cf9ad-d84a-4e52-ba25-d0490bcb9a77 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:34.953467 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:34.954420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:34.954683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:34.959146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] PATH_INFO: `/v3/projects/461a578ac4d44ffc9284d56a09d6c18e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:34.960579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=461a578ac4d44ffc9284d56a09d6c18e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:34.963208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:35.120633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:35.122402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '461a578ac4d44ffc9284d56a09d6c18e', 'request_id': 'req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:35.123782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac07db4664c348f08c85be2a6ae8d210'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:35.130511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '461a578ac4d44ffc9284d56a09d6c18e', 'request_id': 'req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:35.132106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac07db4664c348f08c85be2a6ae8d210'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:35.223532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac07db4664c348f08c85be2a6ae8d210'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:35.224632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac07db4664c348f08c85be2a6ae8d210'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:35.228940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a9c73889-107d-4b82-94c9-34b83c0445e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac07db4664c348f08c85be2a6ae8d210'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:35.248801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1072/2118] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:34 2022] DELETE /identity/v3/users/ac07db4664c348f08c85be2a6ae8d210 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:35.253607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c953b-2cc9-4ea9-9136-fe2a8104f240 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:35.253956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c953b-2cc9-4ea9-9136-fe2a8104f240 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:35.254267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c953b-2cc9-4ea9-9136-fe2a8104f240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:35.273981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-136c953b-2cc9-4ea9-9136-fe2a8104f240 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:35.275270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-136c953b-2cc9-4ea9-9136-fe2a8104f240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:35.276832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '461a578ac4d44ffc9284d56a09d6c18e', 'request_id': 'req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:35.288921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] The token cache is being invalidate because project 461a578ac4d44ffc9284d56a09d6c18e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:35.289318 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:35.290056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a102b8a1-f868-4c2c-bdeb-a4d0b9fa33b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:35.291853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1047/2119] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:34 2022] DELETE /identity/v3/projects/461a578ac4d44ffc9284d56a09d6c18e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:35.296538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2cf49-8e5a-4655-8d5c-1f6043600921 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:35.296822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2cf49-8e5a-4655-8d5c-1f6043600921 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:35.297096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2cf49-8e5a-4655-8d5c-1f6043600921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:35.316320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4fe2cf49-8e5a-4655-8d5c-1f6043600921 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:35.317536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4fe2cf49-8e5a-4655-8d5c-1f6043600921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:35.346439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1073/2120] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:35.357251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c953b-2cc9-4ea9-9136-fe2a8104f240 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:35.358924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-136c953b-2cc9-4ea9-9136-fe2a8104f240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:35.399276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1048/2121] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:35.407952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c2f56-30c7-4401-a74f-1f61be6a5bd7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:35.408337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c2f56-30c7-4401-a74f-1f61be6a5bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:35.408967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c2f56-30c7-4401-a74f-1f61be6a5bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:35.430273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9d6c2f56-30c7-4401-a74f-1f61be6a5bd7 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:35.431857 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9d6c2f56-30c7-4401-a74f-1f61be6a5bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:35.442730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c953b-2cc9-4ea9-9136-fe2a8104f240 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:35.443477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1049/2122] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:35.446834 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72e601-d04f-4619-800a-7023797591a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:35.447451 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c2f56-30c7-4401-a74f-1f61be6a5bd7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:35.447687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e601-d04f-4619-800a-7023797591a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:35.448055 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e601-d04f-4619-800a-7023797591a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:35.448463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e601-d04f-4619-800a-7023797591a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:35.459279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c2f56-30c7-4401-a74f-1f61be6a5bd7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:35.462467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72e601-d04f-4619-800a-7023797591a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:35.463217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e1a4f-0d36-4672-bb26-62d11acc8fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:35.463933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72e601-d04f-4619-800a-7023797591a7 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:35.464056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e1a4f-0d36-4672-bb26-62d11acc8fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:35.464509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e1a4f-0d36-4672-bb26-62d11acc8fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:35.464874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e1a4f-0d36-4672-bb26-62d11acc8fd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:35.465811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e1a4f-0d36-4672-bb26-62d11acc8fd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:35.466270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e1a4f-0d36-4672-bb26-62d11acc8fd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:35.468138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e1a4f-0d36-4672-bb26-62d11acc8fd4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:35.486325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1050/2123] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:07:35.492159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e1a4f-0d36-4672-bb26-62d11acc8fd4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:35.499602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1074/2124] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:35.504302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e1a4f-0d36-4672-bb26-62d11acc8fd4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:35.509284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7f46c-8e02-488c-ae52-52ba85c21827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:35.510609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7f46c-8e02-488c-ae52-52ba85c21827 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:35.510952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7f46c-8e02-488c-ae52-52ba85c21827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:35.511276 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7f46c-8e02-488c-ae52-52ba85c21827 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:35.512115 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7f46c-8e02-488c-ae52-52ba85c21827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:35.514355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7f46c-8e02-488c-ae52-52ba85c21827 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:35.643503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1051/2125] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:35 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:35.644715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72e601-d04f-4619-800a-7023797591a7 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:35.654635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72e601-d04f-4619-800a-7023797591a7 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:35.729587 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e3a30-12b8-47e9-bd6f-5456fcf5162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:35.731093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e3a30-12b8-47e9-bd6f-5456fcf5162c admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:35.731729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e3a30-12b8-47e9-bd6f-5456fcf5162c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:35.731899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e3a30-12b8-47e9-bd6f-5456fcf5162c admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:35.732702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e3a30-12b8-47e9-bd6f-5456fcf5162c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:35.749163 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e3a30-12b8-47e9-bd6f-5456fcf5162c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:35.834550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1075/2126] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:35 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:35.838628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7f46c-8e02-488c-ae52-52ba85c21827 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:35.849606 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7f46c-8e02-488c-ae52-52ba85c21827 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:35.858288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e1b30-5322-48e9-8529-863c8ac62b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:35.859036 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1b30-5322-48e9-8529-863c8ac62b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:35.859391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1b30-5322-48e9-8529-863c8ac62b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:35.865977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1b30-5322-48e9-8529-863c8ac62b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:35.866978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e1b30-5322-48e9-8529-863c8ac62b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:35.869959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e1b30-5322-48e9-8529-863c8ac62b8b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:35.872438 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:35.885136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1052/2127] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:35.889604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e3a30-12b8-47e9-bd6f-5456fcf5162c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:35.901668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e3a30-12b8-47e9-bd6f-5456fcf5162c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:35.906731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ab8d4-c964-496c-90f9-cfb60bdd5585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:35.907401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ab8d4-c964-496c-90f9-cfb60bdd5585 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:35.907686 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ab8d4-c964-496c-90f9-cfb60bdd5585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:35.907965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ab8d4-c964-496c-90f9-cfb60bdd5585 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:35.909050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ab8d4-c964-496c-90f9-cfb60bdd5585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:35.910873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ab8d4-c964-496c-90f9-cfb60bdd5585 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:35.924863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1076/2128] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:35.927722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:35.939375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e1b30-5322-48e9-8529-863c8ac62b8b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:35.949437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e1b30-5322-48e9-8529-863c8ac62b8b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:35.956314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd2e62-59e6-4892-a7e8-fcfc47c229a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:35.962802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd2e62-59e6-4892-a7e8-fcfc47c229a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:35.962802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd2e62-59e6-4892-a7e8-fcfc47c229a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:35.962802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd2e62-59e6-4892-a7e8-fcfc47c229a4 admin admin] PATH_INFO: `/v3/projects/119065e59b3a424792e33a9e9ce102e4/users/46b1196a54094ff49ee8afabb758281c/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:35.963812 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ab8d4-c964-496c-90f9-cfb60bdd5585 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:35.965595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2d2ab8d4-c964-496c-90f9-cfb60bdd5585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:35.976152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd2e62-59e6-4892-a7e8-fcfc47c229a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=119065e59b3a424792e33a9e9ce102e4, user_id=46b1196a54094ff49ee8afabb758281c, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:35.978869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd2e62-59e6-4892-a7e8-fcfc47c229a4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:36.028938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ab8d4-c964-496c-90f9-cfb60bdd5585 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:36.032755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3e3bd-7833-42a3-b8b3-ece9f5351298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:36.034128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3e3bd-7833-42a3-b8b3-ece9f5351298 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:36.035158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3e3bd-7833-42a3-b8b3-ece9f5351298 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:36.035459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3e3bd-7833-42a3-b8b3-ece9f5351298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:36.037662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ccc3e3bd-7833-42a3-b8b3-ece9f5351298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:36.044210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1053/2129] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:35 2022] PUT /identity/v3/projects/119065e59b3a424792e33a9e9ce102e4/users/46b1196a54094ff49ee8afabb758281c/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:36.046328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd2e62-59e6-4892-a7e8-fcfc47c229a4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:36.048030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fcfd2e62-59e6-4892-a7e8-fcfc47c229a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:36.131077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3e3bd-7833-42a3-b8b3-ece9f5351298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:36.131415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd2e62-59e6-4892-a7e8-fcfc47c229a4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:36.132759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3e3bd-7833-42a3-b8b3-ece9f5351298 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:36.136376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:36.137247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:36.139369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:36.139369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] PATH_INFO: `/v3/projects/d5a484cba13e49698f12fe9f0fc8aab8` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:36.139369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a484cba13e49698f12fe9f0fc8aab8)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:36.141401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:36.144438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1077/2130] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:36.152318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900d569-d5ac-42dc-9392-d36a7fee2d67 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:36.152867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900d569-d5ac-42dc-9392-d36a7fee2d67 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:36.153157 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900d569-d5ac-42dc-9392-d36a7fee2d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:36.193247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0900d569-d5ac-42dc-9392-d36a7fee2d67 None None] MFA Rules not processed for user `46b1196a54094ff49ee8afabb758281c`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:36.194436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0900d569-d5ac-42dc-9392-d36a7fee2d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:36.251434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a484cba13e49698f12fe9f0fc8aab8', 'request_id': 'req-69731d30-5f6c-427d-9998-decdaadb2123'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:36.258887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a484cba13e49698f12fe9f0fc8aab8', 'request_id': 'req-69731d30-5f6c-427d-9998-decdaadb2123'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:36.274621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1078/2131] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:36.279001 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487e3fc-1e7a-4f80-8186-c94c82c978d8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:36.279001 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487e3fc-1e7a-4f80-8186-c94c82c978d8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:36.279001 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487e3fc-1e7a-4f80-8186-c94c82c978d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:36.304715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b487e3fc-1e7a-4f80-8186-c94c82c978d8 None None] MFA Rules not processed for user `f9dba100ffe24659b2b42e8617c3dc14`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:36.305924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b487e3fc-1e7a-4f80-8186-c94c82c978d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:36.345143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a484cba13e49698f12fe9f0fc8aab8', 'request_id': 'req-69731d30-5f6c-427d-9998-decdaadb2123'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:36.359265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] The token cache is being invalidate because project d5a484cba13e49698f12fe9f0fc8aab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:36.359554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:36.360280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:36.363114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1054/2132] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:36 2022] DELETE /identity/v3/projects/d5a484cba13e49698f12fe9f0fc8aab8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:36.363898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:36.365244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:36.388606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1079/2133] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:36.389485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487e3fc-1e7a-4f80-8186-c94c82c978d8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:36.395230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b487e3fc-1e7a-4f80-8186-c94c82c978d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:36.453512 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69731d30-5f6c-427d-9998-decdaadb2123 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:36.459971 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69457-5e34-4781-81ef-218ce44503d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:36.460659 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69457-5e34-4781-81ef-218ce44503d1 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:36.460964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69457-5e34-4781-81ef-218ce44503d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:36.461246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69457-5e34-4781-81ef-218ce44503d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:36.463134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dab69457-5e34-4781-81ef-218ce44503d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:36.475113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487e3fc-1e7a-4f80-8186-c94c82c978d8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:36.480286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:36.481225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:36.481535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:36.481847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] PATH_INFO: `/v3/users/1c0c9a340a3c4fa694e97672628857e9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:36.484044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c0c9a340a3c4fa694e97672628857e9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:36.485318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:36.534622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab69457-5e34-4781-81ef-218ce44503d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:36.536214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab69457-5e34-4781-81ef-218ce44503d1 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:36.545420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1055/2134] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:36.546246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69457-5e34-4781-81ef-218ce44503d1 service nova] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:36.547565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dab69457-5e34-4781-81ef-218ce44503d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:36.616255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69457-5e34-4781-81ef-218ce44503d1 service nova] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:36.620069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4999f9-c0b0-4eef-8168-ca3488073bed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:36.620713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4999f9-c0b0-4eef-8168-ca3488073bed service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:36.620990 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4999f9-c0b0-4eef-8168-ca3488073bed service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:36.621732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4999f9-c0b0-4eef-8168-ca3488073bed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:36.623161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4e4999f9-c0b0-4eef-8168-ca3488073bed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:36.686539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4999f9-c0b0-4eef-8168-ca3488073bed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:36.688071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4999f9-c0b0-4eef-8168-ca3488073bed service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:36.700077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1056/2135] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:36.745126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c0c9a340a3c4fa694e97672628857e9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:36.760236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c0c9a340a3c4fa694e97672628857e9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:36.787407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:36.808782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c0c9a340a3c4fa694e97672628857e9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:36.809501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c0c9a340a3c4fa694e97672628857e9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:36.810317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8ccf01e-bf04-4040-81fe-7e5c1fb060ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c0c9a340a3c4fa694e97672628857e9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:36.849566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1080/2136] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:36 2022] DELETE /identity/v3/users/1c0c9a340a3c4fa694e97672628857e9 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:37.084473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:37.261244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4999f9-c0b0-4eef-8168-ca3488073bed service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:37.301179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4999f9-c0b0-4eef-8168-ca3488073bed service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:37.370248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:37.371115 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:37.371394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:37.371673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] PATH_INFO: `/v3/projects/039f4437d496490f86be87589f53e3c7` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:37.372862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=039f4437d496490f86be87589f53e3c7)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:37.374808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:37.560236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '039f4437d496490f86be87589f53e3c7', 'request_id': 'req-843d4e8e-4772-4a97-880e-0aa494425ab5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:37.566917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '039f4437d496490f86be87589f53e3c7', 'request_id': 'req-843d4e8e-4772-4a97-880e-0aa494425ab5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:37.692884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '039f4437d496490f86be87589f53e3c7', 'request_id': 'req-843d4e8e-4772-4a97-880e-0aa494425ab5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:37.704612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] The token cache is being invalidate because project 039f4437d496490f86be87589f53e3c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:37.704957 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:37.705858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-843d4e8e-4772-4a97-880e-0aa494425ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:37.707863 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1057/2137] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:37 2022] DELETE /identity/v3/projects/039f4437d496490f86be87589f53e3c7 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:37.712494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50621a47-37e4-4b10-9c21-39af35677732 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:37.712779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50621a47-37e4-4b10-9c21-39af35677732 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:37.713086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50621a47-37e4-4b10-9c21-39af35677732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:37.732359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-50621a47-37e4-4b10-9c21-39af35677732 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:37.733576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-50621a47-37e4-4b10-9c21-39af35677732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:37.793656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1081/2138] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:37.802113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918bdd1-529d-4fd0-ab32-4b39668936ba None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:37.802404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918bdd1-529d-4fd0-ab32-4b39668936ba None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:37.802684 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918bdd1-529d-4fd0-ab32-4b39668936ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:37.820669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c918bdd1-529d-4fd0-ab32-4b39668936ba None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:37.821954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c918bdd1-529d-4fd0-ab32-4b39668936ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:37.830407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1058/2139] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:37.834790 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50621a47-37e4-4b10-9c21-39af35677732 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:37.844257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50621a47-37e4-4b10-9c21-39af35677732 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:37.848895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:37.856123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bba4c0-e48d-470f-9acf-7fb208abcce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:37.856123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bba4c0-e48d-470f-9acf-7fb208abcce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:37.856123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bba4c0-e48d-470f-9acf-7fb208abcce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:37.856123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bba4c0-e48d-470f-9acf-7fb208abcce9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:37.856450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bba4c0-e48d-470f-9acf-7fb208abcce9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:37.856450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bba4c0-e48d-470f-9acf-7fb208abcce9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:37.856450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bba4c0-e48d-470f-9acf-7fb208abcce9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:37.873920 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1082/2140] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:07:37.876453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918bdd1-529d-4fd0-ab32-4b39668936ba None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:37.885787 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918bdd1-529d-4fd0-ab32-4b39668936ba None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:37.890758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce60af7-88c7-4d3c-8411-66628bd45b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:37.891409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce60af7-88c7-4d3c-8411-66628bd45b72 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:37.891689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce60af7-88c7-4d3c-8411-66628bd45b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:37.891961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce60af7-88c7-4d3c-8411-66628bd45b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:37.893249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce60af7-88c7-4d3c-8411-66628bd45b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:37.895171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce60af7-88c7-4d3c-8411-66628bd45b72 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:37.938879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:38.006161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1059/2141] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:37 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:38.007537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bba4c0-e48d-470f-9acf-7fb208abcce9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:38.019015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bba4c0-e48d-470f-9acf-7fb208abcce9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:38.087816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2466219-fe95-487b-8389-38fd08be6021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:38.088710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2466219-fe95-487b-8389-38fd08be6021 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:38.089005 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2466219-fe95-487b-8389-38fd08be6021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:38.089287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2466219-fe95-487b-8389-38fd08be6021 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:38.089979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2466219-fe95-487b-8389-38fd08be6021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:38.091940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2466219-fe95-487b-8389-38fd08be6021 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:38.188933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1083/2142] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:38 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:38.191117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce60af7-88c7-4d3c-8411-66628bd45b72 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:38.201637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce60af7-88c7-4d3c-8411-66628bd45b72 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:38.208593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124cfdf-dc05-4141-a724-74d234648ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:38.209283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124cfdf-dc05-4141-a724-74d234648ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:38.209530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124cfdf-dc05-4141-a724-74d234648ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:38.209803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124cfdf-dc05-4141-a724-74d234648ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:38.210572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124cfdf-dc05-4141-a724-74d234648ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:38.212176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124cfdf-dc05-4141-a724-74d234648ce9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:38.222989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1060/2143] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:38.227514 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2466219-fe95-487b-8389-38fd08be6021 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:38.238074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2466219-fe95-487b-8389-38fd08be6021 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:38.242763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0a642-77bd-4db9-a1b6-dce0623f8c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:38.243417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0a642-77bd-4db9-a1b6-dce0623f8c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:38.243700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0a642-77bd-4db9-a1b6-dce0623f8c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:38.243981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0a642-77bd-4db9-a1b6-dce0623f8c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:38.244705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0a642-77bd-4db9-a1b6-dce0623f8c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:38.246365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0a642-77bd-4db9-a1b6-dce0623f8c75 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:38.257588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1084/2144] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:38.261748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124cfdf-dc05-4141-a724-74d234648ce9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:38.272532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124cfdf-dc05-4141-a724-74d234648ce9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:38.277410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37762f56-212f-42bc-9b25-da7bdb8cd5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:38.278344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37762f56-212f-42bc-9b25-da7bdb8cd5f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:38.278624 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37762f56-212f-42bc-9b25-da7bdb8cd5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:38.278892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37762f56-212f-42bc-9b25-da7bdb8cd5f6 admin admin] PATH_INFO: `/v3/projects/16721ae1330d4a03ac25d25b2ee8541b/users/331e4f7e78284c4bb0b05998d36c0f68/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:38.296462 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37762f56-212f-42bc-9b25-da7bdb8cd5f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16721ae1330d4a03ac25d25b2ee8541b, user_id=331e4f7e78284c4bb0b05998d36c0f68, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:38.299946 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37762f56-212f-42bc-9b25-da7bdb8cd5f6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:38.370348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1061/2145] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:38 2022] PUT /identity/v3/projects/16721ae1330d4a03ac25d25b2ee8541b/users/331e4f7e78284c4bb0b05998d36c0f68/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:38.382277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e39e20-5235-4276-9304-38f10dce4229 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:38.382581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e39e20-5235-4276-9304-38f10dce4229 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:38.382897 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e39e20-5235-4276-9304-38f10dce4229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:38.414602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f5e39e20-5235-4276-9304-38f10dce4229 None None] MFA Rules not processed for user `331e4f7e78284c4bb0b05998d36c0f68`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:38.415890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f5e39e20-5235-4276-9304-38f10dce4229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:38.493636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1085/2146] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:38.497986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37762f56-212f-42bc-9b25-da7bdb8cd5f6 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:38.501241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e39e20-5235-4276-9304-38f10dce4229 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:38.503228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f5e39e20-5235-4276-9304-38f10dce4229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:38.566080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e39e20-5235-4276-9304-38f10dce4229 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:38.566295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37762f56-212f-42bc-9b25-da7bdb8cd5f6 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:38.569598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53cb9d-a80c-4e42-9ebf-648815fde7f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:38.570262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53cb9d-a80c-4e42-9ebf-648815fde7f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:38.570540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53cb9d-a80c-4e42-9ebf-648815fde7f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:38.570698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1259-54a0-4531-a782-eb3d559b595b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:38.570806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53cb9d-a80c-4e42-9ebf-648815fde7f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:38.571398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1259-54a0-4531-a782-eb3d559b595b admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:38.571681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1259-54a0-4531-a782-eb3d559b595b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:38.571958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1259-54a0-4531-a782-eb3d559b595b admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:38.572579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2e53cb9d-a80c-4e42-9ebf-648815fde7f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:38.572682 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee1259-54a0-4531-a782-eb3d559b595b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:38.574569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee1259-54a0-4531-a782-eb3d559b595b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:38.636324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53cb9d-a80c-4e42-9ebf-648815fde7f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:38.637795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53cb9d-a80c-4e42-9ebf-648815fde7f1 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:38.647740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1086/2147] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:07:38.693510 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1062/2148] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:38 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:38.695070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53cb9d-a80c-4e42-9ebf-648815fde7f1 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:38.705760 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53cb9d-a80c-4e42-9ebf-648815fde7f1 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:38.734313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:38.776787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963a181-8a83-4927-9996-171261d418b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:38.777608 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963a181-8a83-4927-9996-171261d418b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:38.777893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963a181-8a83-4927-9996-171261d418b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:38.778221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963a181-8a83-4927-9996-171261d418b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:38.779167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963a181-8a83-4927-9996-171261d418b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:38.781110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963a181-8a83-4927-9996-171261d418b6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:38.915797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1087/2149] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:38 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:38.920090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1259-54a0-4531-a782-eb3d559b595b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:38.931535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1259-54a0-4531-a782-eb3d559b595b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:38.936235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf9790-5d75-4753-8437-21ad375c4bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:38.937251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf9790-5d75-4753-8437-21ad375c4bdb admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:38.937539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf9790-5d75-4753-8437-21ad375c4bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:38.937821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf9790-5d75-4753-8437-21ad375c4bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:38.938583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf9790-5d75-4753-8437-21ad375c4bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:38.941032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf9790-5d75-4753-8437-21ad375c4bdb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:38.942396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:38.952231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1063/2150] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:38.956415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963a181-8a83-4927-9996-171261d418b6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:38.966223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963a181-8a83-4927-9996-171261d418b6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:38.971490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9f12b-fb00-46b1-8b4a-b430f11d6160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:38.972367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9f12b-fb00-46b1-8b4a-b430f11d6160 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:38.972737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9f12b-fb00-46b1-8b4a-b430f11d6160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:38.973032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9f12b-fb00-46b1-8b4a-b430f11d6160 admin admin] PATH_INFO: `/v3/projects/34c819496ff84af0a6fde5d7637be56c/users/ec06ecb3b57d45068cbab3cbbd0a48ce/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:38.988758 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9f12b-fb00-46b1-8b4a-b430f11d6160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c819496ff84af0a6fde5d7637be56c, user_id=ec06ecb3b57d45068cbab3cbbd0a48ce, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:38.991435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9f12b-fb00-46b1-8b4a-b430f11d6160 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:39.102391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1088/2151] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:38 2022] PUT /identity/v3/projects/34c819496ff84af0a6fde5d7637be56c/users/ec06ecb3b57d45068cbab3cbbd0a48ce/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:39.110843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf9790-5d75-4753-8437-21ad375c4bdb admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:39.126218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf9790-5d75-4753-8437-21ad375c4bdb admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:39.204614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a1b1a-ca2c-46fa-835e-1ecdcb0be56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:39.205312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a1b1a-ca2c-46fa-835e-1ecdcb0be56e admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:39.205864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a1b1a-ca2c-46fa-835e-1ecdcb0be56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:39.206188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a1b1a-ca2c-46fa-835e-1ecdcb0be56e admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:39.207004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a1b1a-ca2c-46fa-835e-1ecdcb0be56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:39.208748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a1b1a-ca2c-46fa-835e-1ecdcb0be56e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:39.224410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1064/2152] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:39.227956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9f12b-fb00-46b1-8b4a-b430f11d6160 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:39.239326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9f12b-fb00-46b1-8b4a-b430f11d6160 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:39.244006 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d403bc3-e848-4c18-9f86-ad7404532df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:39.244890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d403bc3-e848-4c18-9f86-ad7404532df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:39.245168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d403bc3-e848-4c18-9f86-ad7404532df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:39.245438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d403bc3-e848-4c18-9f86-ad7404532df0 admin admin] PATH_INFO: `/v3/projects/34c819496ff84af0a6fde5d7637be56c/users/ec06ecb3b57d45068cbab3cbbd0a48ce/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:39.247489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d403bc3-e848-4c18-9f86-ad7404532df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c819496ff84af0a6fde5d7637be56c, user_id=ec06ecb3b57d45068cbab3cbbd0a48ce, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:39.250223 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d403bc3-e848-4c18-9f86-ad7404532df0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:39.261355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1089/2153] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:39 2022] PUT /identity/v3/projects/34c819496ff84af0a6fde5d7637be56c/users/ec06ecb3b57d45068cbab3cbbd0a48ce/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:39.272594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88196729-3ce2-4b92-98bc-9ee614dc9108 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:39.273058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88196729-3ce2-4b92-98bc-9ee614dc9108 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:39.273151 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88196729-3ce2-4b92-98bc-9ee614dc9108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:39.299691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-88196729-3ce2-4b92-98bc-9ee614dc9108 None None] MFA Rules not processed for user `ec06ecb3b57d45068cbab3cbbd0a48ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:39.300916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-88196729-3ce2-4b92-98bc-9ee614dc9108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:39.385400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1065/2154] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:39.396752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d403bc3-e848-4c18-9f86-ad7404532df0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:39.428153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:39.456530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d403bc3-e848-4c18-9f86-ad7404532df0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:39.460216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d5773-a280-4d0a-b5fe-6ed608d8feeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:39.460856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d5773-a280-4d0a-b5fe-6ed608d8feeb service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:39.461160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d5773-a280-4d0a-b5fe-6ed608d8feeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:39.461409 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d5773-a280-4d0a-b5fe-6ed608d8feeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:39.520957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d5773-a280-4d0a-b5fe-6ed608d8feeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:39.522465 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d5773-a280-4d0a-b5fe-6ed608d8feeb service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:39.531561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1090/2155] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:39.854329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88196729-3ce2-4b92-98bc-9ee614dc9108 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:39.855602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-88196729-3ce2-4b92-98bc-9ee614dc9108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:39.920618 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88196729-3ce2-4b92-98bc-9ee614dc9108 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:39.925866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:39.926776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:39.927061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:39.927337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] PATH_INFO: `/v3/users/46b1196a54094ff49ee8afabb758281c` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:39.928549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b1196a54094ff49ee8afabb758281c)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:39.930524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:39.951243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:40.247063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b1196a54094ff49ee8afabb758281c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:40.255111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b1196a54094ff49ee8afabb758281c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:40.268601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b1196a54094ff49ee8afabb758281c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:40.345585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b1196a54094ff49ee8afabb758281c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:40.346307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b1196a54094ff49ee8afabb758281c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:40.354079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1066/2156] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:39 2022] DELETE /identity/v3/users/46b1196a54094ff49ee8afabb758281c => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:40.358558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e213aa1-96d3-4ff3-a55c-9372ca150059 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:40.359015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e213aa1-96d3-4ff3-a55c-9372ca150059 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:40.359108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e213aa1-96d3-4ff3-a55c-9372ca150059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:40.379000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4e213aa1-96d3-4ff3-a55c-9372ca150059 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:40.380231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4e213aa1-96d3-4ff3-a55c-9372ca150059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:40.468115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1091/2157] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:40.478391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:40.479983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:40.547951 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739984c7-e3d7-44b9-ae7f-7cd8e47eddd5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:40.551898 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a34d3a-eaaa-4965-bae9-fc64f924be53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:40.552547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a34d3a-eaaa-4965-bae9-fc64f924be53 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:40.552832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a34d3a-eaaa-4965-bae9-fc64f924be53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:40.553109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a34d3a-eaaa-4965-bae9-fc64f924be53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:40.562742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a34d3a-eaaa-4965-bae9-fc64f924be53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:40.564202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a34d3a-eaaa-4965-bae9-fc64f924be53 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:40.574424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1067/2158] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:40.900724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e213aa1-96d3-4ff3-a55c-9372ca150059 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:40.902481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4e213aa1-96d3-4ff3-a55c-9372ca150059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:40.915551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e213aa1-96d3-4ff3-a55c-9372ca150059 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:40.920919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:40.921813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:40.922131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:40.922410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] PATH_INFO: `/v3/users/bb72de6f83f24709987b2a3868975af8` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:40.923614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb72de6f83f24709987b2a3868975af8)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:40.926117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:41.043991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a34d3a-eaaa-4965-bae9-fc64f924be53 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:41.045486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-09a34d3a-eaaa-4965-bae9-fc64f924be53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:41.060347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a34d3a-eaaa-4965-bae9-fc64f924be53 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:41.066216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:41.068060 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.068060 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.068277 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] PATH_INFO: `/v3/projects/119065e59b3a424792e33a9e9ce102e4` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.069412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=119065e59b3a424792e33a9e9ce102e4)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:41.072286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:41.176739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb72de6f83f24709987b2a3868975af8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:41.180764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '119065e59b3a424792e33a9e9ce102e4', 'request_id': 'req-482b5b87-8dda-41bb-98ac-1c65bb663ca0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:41.183830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb72de6f83f24709987b2a3868975af8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:41.186757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '119065e59b3a424792e33a9e9ce102e4', 'request_id': 'req-482b5b87-8dda-41bb-98ac-1c65bb663ca0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:41.286796 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb72de6f83f24709987b2a3868975af8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:41.287334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb72de6f83f24709987b2a3868975af8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:41.287939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-233ae4f8-44ae-4e07-a2d4-344772742881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb72de6f83f24709987b2a3868975af8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:41.296327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:41.307062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1092/2159] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:40 2022] DELETE /identity/v3/users/bb72de6f83f24709987b2a3868975af8 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:41.311452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc2db2-f95e-4183-b83b-5ff2700e9ba8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.311741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc2db2-f95e-4183-b83b-5ff2700e9ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.312015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc2db2-f95e-4183-b83b-5ff2700e9ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.329900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8edc2db2-f95e-4183-b83b-5ff2700e9ba8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:41.331118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8edc2db2-f95e-4183-b83b-5ff2700e9ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:41.345016 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '119065e59b3a424792e33a9e9ce102e4', 'request_id': 'req-482b5b87-8dda-41bb-98ac-1c65bb663ca0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:41.359174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] The token cache is being invalidate because project 119065e59b3a424792e33a9e9ce102e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:41.359769 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:41.360104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-482b5b87-8dda-41bb-98ac-1c65bb663ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:41.361938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1068/2160] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:41 2022] DELETE /identity/v3/projects/119065e59b3a424792e33a9e9ce102e4 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:41.367476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b04368-122a-4253-a133-4cb74817ee24 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.367760 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b04368-122a-4253-a133-4cb74817ee24 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.368332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b04368-122a-4253-a133-4cb74817ee24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.386538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-94b04368-122a-4253-a133-4cb74817ee24 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:41.387721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-94b04368-122a-4253-a133-4cb74817ee24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:41.395364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1093/2161] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:41.396321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1069/2162] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:41.404865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b04368-122a-4253-a133-4cb74817ee24 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:41.405376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2bb60-8c78-4fa2-806a-2452bc4ab80b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.405695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2bb60-8c78-4fa2-806a-2452bc4ab80b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.409083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2bb60-8c78-4fa2-806a-2452bc4ab80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.409758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-94b04368-122a-4253-a133-4cb74817ee24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:41.426255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-17b2bb60-8c78-4fa2-806a-2452bc4ab80b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:41.427495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-17b2bb60-8c78-4fa2-806a-2452bc4ab80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:41.429252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:41.438304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1094/2163] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:41.443299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2bb60-8c78-4fa2-806a-2452bc4ab80b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:41.457903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2bb60-8c78-4fa2-806a-2452bc4ab80b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:41.463893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcf183-5c82-4a1d-ad72-5be9090a5b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:41.465632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcf183-5c82-4a1d-ad72-5be9090a5b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.465632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcf183-5c82-4a1d-ad72-5be9090a5b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.465632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcf183-5c82-4a1d-ad72-5be9090a5b03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.466452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcf183-5c82-4a1d-ad72-5be9090a5b03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:41.467180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcf183-5c82-4a1d-ad72-5be9090a5b03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:41.469178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcf183-5c82-4a1d-ad72-5be9090a5b03 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:41.488717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1095/2164] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:07:41.489737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b04368-122a-4253-a133-4cb74817ee24 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:41.490293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcf183-5c82-4a1d-ad72-5be9090a5b03 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:41.496841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d58f8-4f44-4764-ae01-5ad9d79e928e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:41.498904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d58f8-4f44-4764-ae01-5ad9d79e928e service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.499333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d58f8-4f44-4764-ae01-5ad9d79e928e service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.499789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d58f8-4f44-4764-ae01-5ad9d79e928e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.504463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcf183-5c82-4a1d-ad72-5be9090a5b03 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:41.511033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca2db5-b949-4dd7-89a9-077d175102b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:41.512025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca2db5-b949-4dd7-89a9-077d175102b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.512519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca2db5-b949-4dd7-89a9-077d175102b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.513004 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca2db5-b949-4dd7-89a9-077d175102b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.514506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca2db5-b949-4dd7-89a9-077d175102b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:41.519185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d58f8-4f44-4764-ae01-5ad9d79e928e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:41.519961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca2db5-b949-4dd7-89a9-077d175102b9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:41.521038 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d58f8-4f44-4764-ae01-5ad9d79e928e service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:41.534355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1070/2165] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:41.656720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1096/2166] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:41 2022] POST /identity/v3/projects => generated 393 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:41.661601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d58f8-4f44-4764-ae01-5ad9d79e928e service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:41.671304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d58f8-4f44-4764-ae01-5ad9d79e928e service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:41.729603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ef018-667b-4d93-9eda-f8b02077cb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:41.730310 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ef018-667b-4d93-9eda-f8b02077cb08 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.730594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ef018-667b-4d93-9eda-f8b02077cb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.730905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ef018-667b-4d93-9eda-f8b02077cb08 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.731605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ef018-667b-4d93-9eda-f8b02077cb08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:41.733696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ef018-667b-4d93-9eda-f8b02077cb08 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:41.836499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1071/2167] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:41 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:41.840851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca2db5-b949-4dd7-89a9-077d175102b9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:41.852507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca2db5-b949-4dd7-89a9-077d175102b9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:41.857509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161fce1-cbf6-4f66-88d6-cdcec4a6c7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:41.858299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161fce1-cbf6-4f66-88d6-cdcec4a6c7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.860469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161fce1-cbf6-4f66-88d6-cdcec4a6c7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.860469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161fce1-cbf6-4f66-88d6-cdcec4a6c7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.860469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161fce1-cbf6-4f66-88d6-cdcec4a6c7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:41.861231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161fce1-cbf6-4f66-88d6-cdcec4a6c7f7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:41.871747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1097/2168] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:41.875796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ef018-667b-4d93-9eda-f8b02077cb08 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:41.886631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ef018-667b-4d93-9eda-f8b02077cb08 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:41.892390 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd898dd-5fd8-483a-9e47-0c0d5c6ebce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:41.893043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd898dd-5fd8-483a-9e47-0c0d5c6ebce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.893325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd898dd-5fd8-483a-9e47-0c0d5c6ebce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.894155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd898dd-5fd8-483a-9e47-0c0d5c6ebce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.894892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd898dd-5fd8-483a-9e47-0c0d5c6ebce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:41.896480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd898dd-5fd8-483a-9e47-0c0d5c6ebce7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:41.909355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1072/2169] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:41.914125 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161fce1-cbf6-4f66-88d6-cdcec4a6c7f7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:41.924509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161fce1-cbf6-4f66-88d6-cdcec4a6c7f7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:41.929515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cc193-8afb-41cc-8f65-8992766b4f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:41.930429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cc193-8afb-41cc-8f65-8992766b4f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:41.930713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cc193-8afb-41cc-8f65-8992766b4f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:41.930989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cc193-8afb-41cc-8f65-8992766b4f87 admin admin] PATH_INFO: `/v3/projects/e39b8b2c51b24597928941a3b81cd7f4/users/0012b9ce76b946a5ae3beedf65559f3f/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:41.947237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cc193-8afb-41cc-8f65-8992766b4f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39b8b2c51b24597928941a3b81cd7f4, user_id=0012b9ce76b946a5ae3beedf65559f3f, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:41.949917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cc193-8afb-41cc-8f65-8992766b4f87 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:41.984552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd898dd-5fd8-483a-9e47-0c0d5c6ebce7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:41.984552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7bd898dd-5fd8-483a-9e47-0c0d5c6ebce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:41.990879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd898dd-5fd8-483a-9e47-0c0d5c6ebce7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:41.996558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:42.004461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:42.004461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:42.004461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] PATH_INFO: `/v3/projects/621ab3540def48f2ac9ab581ef09d557` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:42.004461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=621ab3540def48f2ac9ab581ef09d557)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:42.004461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:42.049057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1098/2170] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:41 2022] PUT /identity/v3/projects/e39b8b2c51b24597928941a3b81cd7f4/users/0012b9ce76b946a5ae3beedf65559f3f/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:42.061119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572cead-c1c7-4503-8fc8-eda263bbce78 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:42.061412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572cead-c1c7-4503-8fc8-eda263bbce78 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:42.061687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572cead-c1c7-4503-8fc8-eda263bbce78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:42.087372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7572cead-c1c7-4503-8fc8-eda263bbce78 None None] MFA Rules not processed for user `0012b9ce76b946a5ae3beedf65559f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:42.088560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7572cead-c1c7-4503-8fc8-eda263bbce78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:42.154948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621ab3540def48f2ac9ab581ef09d557', 'request_id': 'req-50dd20fe-c8de-424f-833d-166a07f0a395'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:42.165850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621ab3540def48f2ac9ab581ef09d557', 'request_id': 'req-50dd20fe-c8de-424f-833d-166a07f0a395'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:42.195702 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1099/2171] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:42.203304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572cead-c1c7-4503-8fc8-eda263bbce78 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:42.210754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:42.249075 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621ab3540def48f2ac9ab581ef09d557', 'request_id': 'req-50dd20fe-c8de-424f-833d-166a07f0a395'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:42.266474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] The token cache is being invalidate because project 621ab3540def48f2ac9ab581ef09d557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:42.267207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:42.267896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:42.273754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1073/2172] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:41 2022] DELETE /identity/v3/projects/621ab3540def48f2ac9ab581ef09d557 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:42.275163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:42.276518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:42.295293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572cead-c1c7-4503-8fc8-eda263bbce78 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:42.299124 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:42.341699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd20fe-c8de-424f-833d-166a07f0a395 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:42.349190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:42.350166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:42.350402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:42.350675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] PATH_INFO: `/v3/users/3da896b6fc764233a1fc331851026870` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:42.351957 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3da896b6fc764233a1fc331851026870)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:42.352565 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f0e6a-b316-488f-8ddd-55d9fe33b03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:42.353225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f0e6a-b316-488f-8ddd-55d9fe33b03d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:42.353510 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f0e6a-b316-488f-8ddd-55d9fe33b03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:42.353785 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f0e6a-b316-488f-8ddd-55d9fe33b03d admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:42.353883 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:42.354506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f0e6a-b316-488f-8ddd-55d9fe33b03d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:42.356473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f0e6a-b316-488f-8ddd-55d9fe33b03d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:42.485108 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1100/2173] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:42 2022] POST /identity/v3/projects => generated 391 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:42.490310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f0e6a-b316-488f-8ddd-55d9fe33b03d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:42.491921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c50f0e6a-b316-488f-8ddd-55d9fe33b03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:42.558109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f0e6a-b316-488f-8ddd-55d9fe33b03d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:42.560810 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4346c-5670-4eb9-b89d-82c7e27a4c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:42.561480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4346c-5670-4eb9-b89d-82c7e27a4c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:42.561767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4346c-5670-4eb9-b89d-82c7e27a4c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:42.562115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4346c-5670-4eb9-b89d-82c7e27a4c8d admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:42.562828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4346c-5670-4eb9-b89d-82c7e27a4c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:42.564769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4346c-5670-4eb9-b89d-82c7e27a4c8d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:42.668823 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1101/2174] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:42 2022] POST /identity/v3/users => generated 302 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:42.669996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4346c-5670-4eb9-b89d-82c7e27a4c8d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:42.678920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3da896b6fc764233a1fc331851026870'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:42.684630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4346c-5670-4eb9-b89d-82c7e27a4c8d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:42.688190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3da896b6fc764233a1fc331851026870'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:42.695977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972e19e-11d5-44bf-851c-08749a3ca7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:42.696652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972e19e-11d5-44bf-851c-08749a3ca7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:42.696963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972e19e-11d5-44bf-851c-08749a3ca7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:42.697243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972e19e-11d5-44bf-851c-08749a3ca7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:42.697976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972e19e-11d5-44bf-851c-08749a3ca7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:42.699868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972e19e-11d5-44bf-851c-08749a3ca7d0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:42.710338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1102/2175] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:07:42.710766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3da896b6fc764233a1fc331851026870'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:42.715369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972e19e-11d5-44bf-851c-08749a3ca7d0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:42.726350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972e19e-11d5-44bf-851c-08749a3ca7d0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:42.730989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09070d6-a7da-4df1-8f50-89734b446396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:42.731898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09070d6-a7da-4df1-8f50-89734b446396 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:42.732177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09070d6-a7da-4df1-8f50-89734b446396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:42.732449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09070d6-a7da-4df1-8f50-89734b446396 admin admin] PATH_INFO: `/v3/projects/db1e18a039674c8d9aa04000cf403309/users/e47c3f01861846dea5d927f413c578fb/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:42.750416 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09070d6-a7da-4df1-8f50-89734b446396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1e18a039674c8d9aa04000cf403309, user_id=e47c3f01861846dea5d927f413c578fb, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:42.753318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09070d6-a7da-4df1-8f50-89734b446396 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:42.774232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3da896b6fc764233a1fc331851026870'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:42.774968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3da896b6fc764233a1fc331851026870'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:42.781481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1074/2176] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:42 2022] DELETE /identity/v3/users/3da896b6fc764233a1fc331851026870 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:42.856385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1103/2177] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:42 2022] PUT /identity/v3/projects/db1e18a039674c8d9aa04000cf403309/users/e47c3f01861846dea5d927f413c578fb/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:42.860849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:42.862335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:42.929870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f937ae-b28e-44bb-8edf-6ba2b83ec865 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:42.986750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797e6de-b2b4-4d5a-8c20-55400c3960d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:42.987402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e6de-b2b4-4d5a-8c20-55400c3960d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:42.987692 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e6de-b2b4-4d5a-8c20-55400c3960d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:42.987967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e6de-b2b4-4d5a-8c20-55400c3960d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:42.988902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797e6de-b2b4-4d5a-8c20-55400c3960d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:42.990581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797e6de-b2b4-4d5a-8c20-55400c3960d7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:43.001843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1075/2178] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:43.006900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09070d6-a7da-4df1-8f50-89734b446396 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:43.019531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09070d6-a7da-4df1-8f50-89734b446396 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:43.023966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960bf65-b4c1-44a3-a91a-8a910ddb9d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:43.024851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960bf65-b4c1-44a3-a91a-8a910ddb9d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:43.025279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960bf65-b4c1-44a3-a91a-8a910ddb9d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:43.025559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960bf65-b4c1-44a3-a91a-8a910ddb9d28 admin admin] PATH_INFO: `/v3/projects/db1e18a039674c8d9aa04000cf403309/users/e47c3f01861846dea5d927f413c578fb/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:43.027567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960bf65-b4c1-44a3-a91a-8a910ddb9d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1e18a039674c8d9aa04000cf403309, user_id=e47c3f01861846dea5d927f413c578fb, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:43.030237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960bf65-b4c1-44a3-a91a-8a910ddb9d28 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:43.041020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1104/2179] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:43 2022] PUT /identity/v3/projects/db1e18a039674c8d9aa04000cf403309/users/e47c3f01861846dea5d927f413c578fb/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:43.049030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:43.052194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e031dd-bb8b-4070-80ff-60ea53cf64b8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:43.052487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e031dd-bb8b-4070-80ff-60ea53cf64b8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:43.052765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e031dd-bb8b-4070-80ff-60ea53cf64b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:43.080096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e2e031dd-bb8b-4070-80ff-60ea53cf64b8 None None] MFA Rules not processed for user `e47c3f01861846dea5d927f413c578fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:43.081374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e2e031dd-bb8b-4070-80ff-60ea53cf64b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:43.100383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:43.153429 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1076/2180] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:07:43.165137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960bf65-b4c1-44a3-a91a-8a910ddb9d28 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:43.166900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b960bf65-b4c1-44a3-a91a-8a910ddb9d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:43.206739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e031dd-bb8b-4070-80ff-60ea53cf64b8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:43.207237 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e2e031dd-bb8b-4070-80ff-60ea53cf64b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:43.230586 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960bf65-b4c1-44a3-a91a-8a910ddb9d28 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:43.234263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8b1c7-da2f-40a9-a0e6-16f02c45650a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:43.234900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8b1c7-da2f-40a9-a0e6-16f02c45650a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:43.235267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8b1c7-da2f-40a9-a0e6-16f02c45650a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:43.235498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8b1c7-da2f-40a9-a0e6-16f02c45650a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:43.249799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8b1c7-da2f-40a9-a0e6-16f02c45650a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:43.251311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8b1c7-da2f-40a9-a0e6-16f02c45650a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:43.262830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1105/2181] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:43.283308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e031dd-bb8b-4070-80ff-60ea53cf64b8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:43.290282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:43.292448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:43.292448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:43.292448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] PATH_INFO: `/v3/projects/97dcd1788b544599aa6d19e343c0b4cb` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:43.293557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] RBAC: Authorizing `identity:delete_project(project_id=97dcd1788b544599aa6d19e343c0b4cb)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:43.295801 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:43.316982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8b1c7-da2f-40a9-a0e6-16f02c45650a service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:43.327554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8b1c7-da2f-40a9-a0e6-16f02c45650a service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:43.331612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7bc46-f654-48bc-aa52-33aa52dfaaba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:43.332209 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7bc46-f654-48bc-aa52-33aa52dfaaba service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:43.332520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7bc46-f654-48bc-aa52-33aa52dfaaba service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:43.332781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7bc46-f654-48bc-aa52-33aa52dfaaba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:43.334745 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-14b7bc46-f654-48bc-aa52-33aa52dfaaba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:43.398934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7bc46-f654-48bc-aa52-33aa52dfaaba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:43.400382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7bc46-f654-48bc-aa52-33aa52dfaaba service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:43.410575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1106/2182] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:43.423109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97dcd1788b544599aa6d19e343c0b4cb', 'request_id': 'req-05d9c100-e268-469d-99f9-e56df8b0944d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:43.430566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97dcd1788b544599aa6d19e343c0b4cb', 'request_id': 'req-05d9c100-e268-469d-99f9-e56df8b0944d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:43.505575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97dcd1788b544599aa6d19e343c0b4cb', 'request_id': 'req-05d9c100-e268-469d-99f9-e56df8b0944d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:43.518079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] The token cache is being invalidate because project 97dcd1788b544599aa6d19e343c0b4cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:43.518441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:43.519794 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:43.521367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1077/2183] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:43 2022] DELETE /identity/v3/projects/97dcd1788b544599aa6d19e343c0b4cb => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:43.525682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7bc46-f654-48bc-aa52-33aa52dfaaba service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:43.537352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7bc46-f654-48bc-aa52-33aa52dfaaba service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:43.595856 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:43.596765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:43.598162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:43.598162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] PATH_INFO: `/v3/users/7c658d5002b6452ba2d5c7b914d1a869` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:43.598783 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c658d5002b6452ba2d5c7b914d1a869)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:43.600725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:43.623850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:43.826727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c658d5002b6452ba2d5c7b914d1a869'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:43.834836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c658d5002b6452ba2d5c7b914d1a869'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:43.869976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:43.898344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c658d5002b6452ba2d5c7b914d1a869'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:43.898990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c658d5002b6452ba2d5c7b914d1a869'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:43.899611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a67216f2-4452-4f89-aab9-fba106c4148c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c658d5002b6452ba2d5c7b914d1a869'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:43.942467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1107/2184] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:43 2022] DELETE /identity/v3/users/7c658d5002b6452ba2d5c7b914d1a869 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:44.381648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:44.383138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:44.399106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9c100-e268-469d-99f9-e56df8b0944d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:44.455635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:44.456592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:44.456871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:44.457143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] PATH_INFO: `/v3/projects/cdc6f8bff7c54f2bb14f7cf2a7c538e6` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:44.458350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc6f8bff7c54f2bb14f7cf2a7c538e6)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:44.460239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:44.579557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc6f8bff7c54f2bb14f7cf2a7c538e6', 'request_id': 'req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:44.586179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc6f8bff7c54f2bb14f7cf2a7c538e6', 'request_id': 'req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:44.654355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc6f8bff7c54f2bb14f7cf2a7c538e6', 'request_id': 'req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:44.667643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] The token cache is being invalidate because project cdc6f8bff7c54f2bb14f7cf2a7c538e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:44.668136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:44.668732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4cab56d0-f0f6-4ec3-84cd-dc4d8fc7adcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:44.671473 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1078/2185] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:44 2022] DELETE /identity/v3/projects/cdc6f8bff7c54f2bb14f7cf2a7c538e6 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:44.677110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a19c-fa14-472c-9203-9d37164a9c47 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:44.677621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a19c-fa14-472c-9203-9d37164a9c47 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:44.677725 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a19c-fa14-472c-9203-9d37164a9c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:44.696271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-34b7a19c-fa14-472c-9203-9d37164a9c47 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:44.697505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-34b7a19c-fa14-472c-9203-9d37164a9c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:44.760647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1108/2186] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:44.769348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd198c-7238-4bca-a3df-9bb698449a04 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:44.769783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd198c-7238-4bca-a3df-9bb698449a04 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:44.770100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd198c-7238-4bca-a3df-9bb698449a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:44.789217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1ecd198c-7238-4bca-a3df-9bb698449a04 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:44.790466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1ecd198c-7238-4bca-a3df-9bb698449a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:44.798288 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1079/2187] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:44.802624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7a19c-fa14-472c-9203-9d37164a9c47 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:44.812248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7a19c-fa14-472c-9203-9d37164a9c47 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:44.816927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0d304-9beb-4141-9d38-50a88c4384de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:44.817583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0d304-9beb-4141-9d38-50a88c4384de admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:44.817861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0d304-9beb-4141-9d38-50a88c4384de admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:44.818176 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0d304-9beb-4141-9d38-50a88c4384de admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:44.818907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0d304-9beb-4141-9d38-50a88c4384de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:44.819213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0d304-9beb-4141-9d38-50a88c4384de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:44.820811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0d304-9beb-4141-9d38-50a88c4384de admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:44.833099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1109/2188] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:07:44.837262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd198c-7238-4bca-a3df-9bb698449a04 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:44.846668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd198c-7238-4bca-a3df-9bb698449a04 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:44.851915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c65c2-c620-4831-aaa5-e12cce1be1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:44.852564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c65c2-c620-4831-aaa5-e12cce1be1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:44.852850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c65c2-c620-4831-aaa5-e12cce1be1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:44.853127 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c65c2-c620-4831-aaa5-e12cce1be1c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:44.853827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c65c2-c620-4831-aaa5-e12cce1be1c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:44.855754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c65c2-c620-4831-aaa5-e12cce1be1c8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:44.933722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1080/2189] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:44 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:44.937855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0d304-9beb-4141-9d38-50a88c4384de admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:44.948811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0d304-9beb-4141-9d38-50a88c4384de admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:45.011672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a498296-204e-4ca1-9761-91be0cbe3a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:45.012348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498296-204e-4ca1-9761-91be0cbe3a57 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:45.012633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498296-204e-4ca1-9761-91be0cbe3a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:45.012917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498296-204e-4ca1-9761-91be0cbe3a57 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:45.013647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a498296-204e-4ca1-9761-91be0cbe3a57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:45.015868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a498296-204e-4ca1-9761-91be0cbe3a57 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:45.113121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1110/2190] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:44 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:45.114635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c65c2-c620-4831-aaa5-e12cce1be1c8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:45.125642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c65c2-c620-4831-aaa5-e12cce1be1c8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:45.131001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a73547-a64f-467f-a209-006118bcb7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:45.131687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a73547-a64f-467f-a209-006118bcb7ad admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:45.137097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a73547-a64f-467f-a209-006118bcb7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:45.137097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a73547-a64f-467f-a209-006118bcb7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:45.137097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a73547-a64f-467f-a209-006118bcb7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:45.137097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a73547-a64f-467f-a209-006118bcb7ad admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:45.146799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1081/2191] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:45.150814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a498296-204e-4ca1-9761-91be0cbe3a57 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:45.161105 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a498296-204e-4ca1-9761-91be0cbe3a57 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:45.165838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f2011-0822-4aa1-a7ce-c28e6e7900e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:45.166635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2011-0822-4aa1-a7ce-c28e6e7900e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:45.166917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2011-0822-4aa1-a7ce-c28e6e7900e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:45.167196 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2011-0822-4aa1-a7ce-c28e6e7900e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:45.167941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f2011-0822-4aa1-a7ce-c28e6e7900e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:45.169531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f2011-0822-4aa1-a7ce-c28e6e7900e6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:45.180039 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1111/2192] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:45.184123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a73547-a64f-467f-a209-006118bcb7ad admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:45.194583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a73547-a64f-467f-a209-006118bcb7ad admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:45.199121 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58986c97-b5d9-4738-8aec-e6d75f2b8b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:45.206708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58986c97-b5d9-4738-8aec-e6d75f2b8b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:45.206708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58986c97-b5d9-4738-8aec-e6d75f2b8b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:45.206708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58986c97-b5d9-4738-8aec-e6d75f2b8b38 admin admin] PATH_INFO: `/v3/projects/be3ff54edaf7439bab9ca81656e94c70/users/1ac8c3e3e99e412bb4a1f7b1506abce2/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:45.215256 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58986c97-b5d9-4738-8aec-e6d75f2b8b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3ff54edaf7439bab9ca81656e94c70, user_id=1ac8c3e3e99e412bb4a1f7b1506abce2, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:45.217936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58986c97-b5d9-4738-8aec-e6d75f2b8b38 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:45.305554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1082/2193] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:45 2022] PUT /identity/v3/projects/be3ff54edaf7439bab9ca81656e94c70/users/1ac8c3e3e99e412bb4a1f7b1506abce2/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:45.312827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77518ad5-f34f-4ee6-bd3a-efbda009dc11 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:45.313116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77518ad5-f34f-4ee6-bd3a-efbda009dc11 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:45.313392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77518ad5-f34f-4ee6-bd3a-efbda009dc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:45.320879 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:45.331838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-77518ad5-f34f-4ee6-bd3a-efbda009dc11 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:45.333119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-77518ad5-f34f-4ee6-bd3a-efbda009dc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:45.393496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1112/2194] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:45.403614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58986c97-b5d9-4738-8aec-e6d75f2b8b38 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:45.405298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-58986c97-b5d9-4738-8aec-e6d75f2b8b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:45.433505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:45.461925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58986c97-b5d9-4738-8aec-e6d75f2b8b38 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:45.465547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be134c-342a-431d-9bf6-bfa5e9509d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:45.466218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be134c-342a-431d-9bf6-bfa5e9509d94 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:45.466503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be134c-342a-431d-9bf6-bfa5e9509d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:45.468696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be134c-342a-431d-9bf6-bfa5e9509d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:45.475796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be134c-342a-431d-9bf6-bfa5e9509d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:45.477205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be134c-342a-431d-9bf6-bfa5e9509d94 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:45.486747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1083/2195] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:48.367640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77518ad5-f34f-4ee6-bd3a-efbda009dc11 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:48.370207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-77518ad5-f34f-4ee6-bd3a-efbda009dc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:48.443500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77518ad5-f34f-4ee6-bd3a-efbda009dc11 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:48.447293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e431a7b-4be4-4daf-978a-9c5a3d3d9465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:48.447951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e431a7b-4be4-4daf-978a-9c5a3d3d9465 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:48.448434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e431a7b-4be4-4daf-978a-9c5a3d3d9465 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:48.448569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e431a7b-4be4-4daf-978a-9c5a3d3d9465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:48.451022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5e431a7b-4be4-4daf-978a-9c5a3d3d9465 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:48.563739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e431a7b-4be4-4daf-978a-9c5a3d3d9465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:48.564167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e431a7b-4be4-4daf-978a-9c5a3d3d9465 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:48.577595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1113/2196] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:51.302899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b3ebd-4771-43b6-811b-83b26cc71e3f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:51.303446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b3ebd-4771-43b6-811b-83b26cc71e3f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:51.303446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b3ebd-4771-43b6-811b-83b26cc71e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:51.351116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bf8b3ebd-4771-43b6-811b-83b26cc71e3f None None] MFA Rules not processed for user `1ac8c3e3e99e412bb4a1f7b1506abce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:51.352361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bf8b3ebd-4771-43b6-811b-83b26cc71e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:51.443594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1084/2197] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:51.447995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e431a7b-4be4-4daf-978a-9c5a3d3d9465 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:51.460686 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e431a7b-4be4-4daf-978a-9c5a3d3d9465 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:51.476391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612489df-ed48-4575-b715-7d5faedb6795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:51.476391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612489df-ed48-4575-b715-7d5faedb6795 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:51.476391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612489df-ed48-4575-b715-7d5faedb6795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:51.476391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612489df-ed48-4575-b715-7d5faedb6795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:51.477084 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612489df-ed48-4575-b715-7d5faedb6795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:51.477084 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612489df-ed48-4575-b715-7d5faedb6795 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:51.583461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1114/2198] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:51 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:51.584926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b3ebd-4771-43b6-811b-83b26cc71e3f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:51.597713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b3ebd-4771-43b6-811b-83b26cc71e3f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:51.670837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e54f1-c49f-4c5f-808d-64735a154fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:51.671517 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e54f1-c49f-4c5f-808d-64735a154fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:51.671791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e54f1-c49f-4c5f-808d-64735a154fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:51.672226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e54f1-c49f-4c5f-808d-64735a154fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:51.672929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e54f1-c49f-4c5f-808d-64735a154fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:51.675754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e54f1-c49f-4c5f-808d-64735a154fa2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:51.780946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612489df-ed48-4575-b715-7d5faedb6795 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:51.781669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1085/2199] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:51 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:51.794469 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612489df-ed48-4575-b715-7d5faedb6795 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:51.799197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991edcd-a760-4cca-91b1-2047329aa4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:51.799889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991edcd-a760-4cca-91b1-2047329aa4ab admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:51.800172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991edcd-a760-4cca-91b1-2047329aa4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:51.800446 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991edcd-a760-4cca-91b1-2047329aa4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:51.801208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991edcd-a760-4cca-91b1-2047329aa4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:51.802854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991edcd-a760-4cca-91b1-2047329aa4ab admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:51.813629 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1115/2200] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:51.819000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e54f1-c49f-4c5f-808d-64735a154fa2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:51.831929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e54f1-c49f-4c5f-808d-64735a154fa2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:51.837849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6c3a1-c888-44f6-b580-d7ad3c2389a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:51.838821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6c3a1-c888-44f6-b580-d7ad3c2389a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:51.839101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6c3a1-c888-44f6-b580-d7ad3c2389a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:51.839374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6c3a1-c888-44f6-b580-d7ad3c2389a2 admin admin] PATH_INFO: `/v3/projects/23ca6e44ed8f47fbbe0960fc3d65025f/users/342a6239810e40aa83db81985af2bfc2/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:51.858704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6c3a1-c888-44f6-b580-d7ad3c2389a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ca6e44ed8f47fbbe0960fc3d65025f, user_id=342a6239810e40aa83db81985af2bfc2, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:51.861376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6c3a1-c888-44f6-b580-d7ad3c2389a2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:51.964966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1086/2201] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:51 2022] PUT /identity/v3/projects/23ca6e44ed8f47fbbe0960fc3d65025f/users/342a6239810e40aa83db81985af2bfc2/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:51.966720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991edcd-a760-4cca-91b1-2047329aa4ab admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:51.979956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991edcd-a760-4cca-91b1-2047329aa4ab admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:52.006500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:52.053225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c9ede-a066-4390-b889-f2a578d65f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:52.053547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c9ede-a066-4390-b889-f2a578d65f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:52.053972 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c9ede-a066-4390-b889-f2a578d65f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:52.054356 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c9ede-a066-4390-b889-f2a578d65f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:52.055208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c9ede-a066-4390-b889-f2a578d65f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:52.057585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c9ede-a066-4390-b889-f2a578d65f20 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:52.070581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1116/2202] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:52.074668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6c3a1-c888-44f6-b580-d7ad3c2389a2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:52.089006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6c3a1-c888-44f6-b580-d7ad3c2389a2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:52.098627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ac605-9dd3-4768-b040-6c13f505b13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:52.099363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ac605-9dd3-4768-b040-6c13f505b13a admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:52.100452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ac605-9dd3-4768-b040-6c13f505b13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:52.100452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ac605-9dd3-4768-b040-6c13f505b13a admin admin] PATH_INFO: `/v3/projects/23ca6e44ed8f47fbbe0960fc3d65025f/users/342a6239810e40aa83db81985af2bfc2/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:52.101995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ac605-9dd3-4768-b040-6c13f505b13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ca6e44ed8f47fbbe0960fc3d65025f, user_id=342a6239810e40aa83db81985af2bfc2, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:52.104819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ac605-9dd3-4768-b040-6c13f505b13a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:52.136217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1087/2203] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:52 2022] PUT /identity/v3/projects/23ca6e44ed8f47fbbe0960fc3d65025f/users/342a6239810e40aa83db81985af2bfc2/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:52.147333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c9ede-a066-4390-b889-f2a578d65f20 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:52.151044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cb5c9ede-a066-4390-b889-f2a578d65f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:52.189951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c9ede-a066-4390-b889-f2a578d65f20 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:52.268694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:52.270148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:52.270591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:52.274467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] PATH_INFO: `/v3/users/0012b9ce76b946a5ae3beedf65559f3f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:52.288524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0012b9ce76b946a5ae3beedf65559f3f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:52.292366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:52.738794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0012b9ce76b946a5ae3beedf65559f3f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:52.754117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0012b9ce76b946a5ae3beedf65559f3f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:52.864151 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0012b9ce76b946a5ae3beedf65559f3f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:52.865120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0012b9ce76b946a5ae3beedf65559f3f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:52.865822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0012b9ce76b946a5ae3beedf65559f3f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:52.890593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1117/2204] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:52 2022] DELETE /identity/v3/users/0012b9ce76b946a5ae3beedf65559f3f => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:52.895997 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90083e34-978a-4e85-8e9e-2a9fa5eac778 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:52.902159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90083e34-978a-4e85-8e9e-2a9fa5eac778 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:52.902159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90083e34-978a-4e85-8e9e-2a9fa5eac778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:52.916122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-90083e34-978a-4e85-8e9e-2a9fa5eac778 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:52.917353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-90083e34-978a-4e85-8e9e-2a9fa5eac778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:52.925347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:53.006203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1088/2205] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:53.013470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:53.017868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:53.068968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:53.086533 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d367ae-249a-4d11-825b-96277a133a8e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:53.090415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cb08f-4966-49b4-857d-0b23bbd8885f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:53.091151 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cb08f-4966-49b4-857d-0b23bbd8885f service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:53.091528 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cb08f-4966-49b4-857d-0b23bbd8885f service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:53.091851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cb08f-4966-49b4-857d-0b23bbd8885f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:53.103377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cb08f-4966-49b4-857d-0b23bbd8885f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:53.105013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cb08f-4966-49b4-857d-0b23bbd8885f service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:53.117655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1118/2206] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:53.580951 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90083e34-978a-4e85-8e9e-2a9fa5eac778 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:53.582968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-90083e34-978a-4e85-8e9e-2a9fa5eac778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:53.604272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90083e34-978a-4e85-8e9e-2a9fa5eac778 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:53.612650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:53.613922 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:53.614408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:53.614841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] PATH_INFO: `/v3/projects/e39b8b2c51b24597928941a3b81cd7f4` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:53.624427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39b8b2c51b24597928941a3b81cd7f4)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:53.626737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:53.744889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39b8b2c51b24597928941a3b81cd7f4', 'request_id': 'req-54697cfb-847b-410c-bd6b-b9a736c6bde1'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:53.750356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39b8b2c51b24597928941a3b81cd7f4', 'request_id': 'req-54697cfb-847b-410c-bd6b-b9a736c6bde1'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:53.853547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39b8b2c51b24597928941a3b81cd7f4', 'request_id': 'req-54697cfb-847b-410c-bd6b-b9a736c6bde1'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:53.868600 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] The token cache is being invalidate because project e39b8b2c51b24597928941a3b81cd7f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:53.869152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:53.870294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:53.872433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1089/2207] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:53 2022] DELETE /identity/v3/projects/e39b8b2c51b24597928941a3b81cd7f4 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:07:53.876334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cb08f-4966-49b4-857d-0b23bbd8885f service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:53.902433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cb08f-4966-49b4-857d-0b23bbd8885f service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:54.009854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:54.010888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:54.011262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:54.011633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] PATH_INFO: `/v3/users/e47c3f01861846dea5d927f413c578fb` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:54.013227 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47c3f01861846dea5d927f413c578fb)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:54.015582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:54.209638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47c3f01861846dea5d927f413c578fb'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:54.228088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47c3f01861846dea5d927f413c578fb'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:54.298864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47c3f01861846dea5d927f413c578fb'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:54.300506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47c3f01861846dea5d927f413c578fb'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:54.300871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-404bc79d-9d1f-409c-92e9-d100ed9f70d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47c3f01861846dea5d927f413c578fb'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:54.354195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1119/2208] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:53 2022] DELETE /identity/v3/users/e47c3f01861846dea5d927f413c578fb => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:54.958508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:54.959976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:54.983317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54697cfb-847b-410c-bd6b-b9a736c6bde1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:55.104322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:55.104322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:55.104322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:55.104322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] PATH_INFO: `/v3/projects/db1e18a039674c8d9aa04000cf403309` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:55.104913 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1e18a039674c8d9aa04000cf403309)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:55.104913 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:55.253637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:55.263880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1e18a039674c8d9aa04000cf403309', 'request_id': 'req-fa613152-8a28-44bb-838b-cfd5af5452ad'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:55.271309 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1e18a039674c8d9aa04000cf403309', 'request_id': 'req-fa613152-8a28-44bb-838b-cfd5af5452ad'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:55.356368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1e18a039674c8d9aa04000cf403309', 'request_id': 'req-fa613152-8a28-44bb-838b-cfd5af5452ad'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:55.372712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] The token cache is being invalidate because project db1e18a039674c8d9aa04000cf403309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:07:55.373035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:55.373680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa613152-8a28-44bb-838b-cfd5af5452ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:55.380690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1090/2209] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:54 2022] DELETE /identity/v3/projects/db1e18a039674c8d9aa04000cf403309 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:55.382283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba4c1c-8689-4acb-b545-610af9fa54c2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:55.382740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba4c1c-8689-4acb-b545-610af9fa54c2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:55.382852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba4c1c-8689-4acb-b545-610af9fa54c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:55.411558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-20ba4c1c-8689-4acb-b545-610af9fa54c2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:55.413025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-20ba4c1c-8689-4acb-b545-610af9fa54c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:55.501770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:55.513762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1120/2210] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:55.525306 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be33b0-e07e-47f7-b878-1dd04f0a1c1e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:55.526752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be33b0-e07e-47f7-b878-1dd04f0a1c1e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:55.530903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be33b0-e07e-47f7-b878-1dd04f0a1c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:55.610454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-09be33b0-e07e-47f7-b878-1dd04f0a1c1e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:55.610454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-09be33b0-e07e-47f7-b878-1dd04f0a1c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:55.610454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1091/2211] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:55.616525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba4c1c-8689-4acb-b545-610af9fa54c2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:55.628523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba4c1c-8689-4acb-b545-610af9fa54c2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:55.641354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46bb45-b2bd-4b2d-a265-b6b72f8c26ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:55.646187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46bb45-b2bd-4b2d-a265-b6b72f8c26ec admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:55.646187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46bb45-b2bd-4b2d-a265-b6b72f8c26ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:55.646187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46bb45-b2bd-4b2d-a265-b6b72f8c26ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:55.646187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46bb45-b2bd-4b2d-a265-b6b72f8c26ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:07:55.646187 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46bb45-b2bd-4b2d-a265-b6b72f8c26ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:55.646762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46bb45-b2bd-4b2d-a265-b6b72f8c26ec admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:55.658947 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1121/2212] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:07:55.667016 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be33b0-e07e-47f7-b878-1dd04f0a1c1e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:55.678132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be33b0-e07e-47f7-b878-1dd04f0a1c1e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:55.683859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1d83d-c7c8-4226-a4e2-95ca811c3256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:55.685074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1d83d-c7c8-4226-a4e2-95ca811c3256 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:55.685358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1d83d-c7c8-4226-a4e2-95ca811c3256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:55.685627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1d83d-c7c8-4226-a4e2-95ca811c3256 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:55.686468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1d83d-c7c8-4226-a4e2-95ca811c3256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:55.688383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1d83d-c7c8-4226-a4e2-95ca811c3256 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:56.039762 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1092/2213] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:07:55 2022] POST /identity/v3/projects => generated 375 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:07:56.043726 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46bb45-b2bd-4b2d-a265-b6b72f8c26ec admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:56.074727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46bb45-b2bd-4b2d-a265-b6b72f8c26ec admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:56.241006 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19becc91-b361-4507-9f53-8c343191e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:56.241733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19becc91-b361-4507-9f53-8c343191e418 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:56.245202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19becc91-b361-4507-9f53-8c343191e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:56.245202 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19becc91-b361-4507-9f53-8c343191e418 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:56.250592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19becc91-b361-4507-9f53-8c343191e418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:56.253413 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19becc91-b361-4507-9f53-8c343191e418 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:56.371506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1122/2214] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:07:56 2022] POST /identity/v3/users => generated 288 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:07:56.374354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1d83d-c7c8-4226-a4e2-95ca811c3256 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:56.386980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1d83d-c7c8-4226-a4e2-95ca811c3256 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:56.392816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6225939-96d5-4b56-b8d2-d96568af28f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:56.393638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6225939-96d5-4b56-b8d2-d96568af28f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:56.394045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6225939-96d5-4b56-b8d2-d96568af28f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:56.396427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6225939-96d5-4b56-b8d2-d96568af28f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:56.396427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6225939-96d5-4b56-b8d2-d96568af28f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:56.397010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6225939-96d5-4b56-b8d2-d96568af28f5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:56.409321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1093/2215] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:56.413599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19becc91-b361-4507-9f53-8c343191e418 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:56.426381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19becc91-b361-4507-9f53-8c343191e418 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:56.431735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6feed3-a047-4fd7-b581-e72a8004b2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:56.432425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6feed3-a047-4fd7-b581-e72a8004b2bf admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:56.432712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6feed3-a047-4fd7-b581-e72a8004b2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:56.432996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6feed3-a047-4fd7-b581-e72a8004b2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:56.433811 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6feed3-a047-4fd7-b581-e72a8004b2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:56.435604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6feed3-a047-4fd7-b581-e72a8004b2bf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:56.448198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1123/2216] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:07:56.452628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6225939-96d5-4b56-b8d2-d96568af28f5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:56.464376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6225939-96d5-4b56-b8d2-d96568af28f5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:56.472295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c58c3-0c44-4bce-a89e-b60927b16593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:56.473957 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c58c3-0c44-4bce-a89e-b60927b16593 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:56.474616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c58c3-0c44-4bce-a89e-b60927b16593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:56.475190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c58c3-0c44-4bce-a89e-b60927b16593 admin admin] PATH_INFO: `/v3/projects/dbb9a286dde7416b92a50b729f45280f/users/f8e8e759196645e3838f52014a1020c2/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:56.494163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c58c3-0c44-4bce-a89e-b60927b16593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb9a286dde7416b92a50b729f45280f, user_id=f8e8e759196645e3838f52014a1020c2, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:56.497597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c58c3-0c44-4bce-a89e-b60927b16593 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:56.609153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1094/2217] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:07:56 2022] PUT /identity/v3/projects/dbb9a286dde7416b92a50b729f45280f/users/f8e8e759196645e3838f52014a1020c2/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:56.624651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d6cf7-37dc-4843-adc8-3c4feb73cbf6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:56.624957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d6cf7-37dc-4843-adc8-3c4feb73cbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:56.625232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d6cf7-37dc-4843-adc8-3c4feb73cbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:56.654364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c28d6cf7-37dc-4843-adc8-3c4feb73cbf6 None None] MFA Rules not processed for user `f8e8e759196645e3838f52014a1020c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:56.655582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c28d6cf7-37dc-4843-adc8-3c4feb73cbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:56.724864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1124/2218] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:56.739293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c58c3-0c44-4bce-a89e-b60927b16593 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:56.741033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7c8c58c3-0c44-4bce-a89e-b60927b16593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:56.835315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c58c3-0c44-4bce-a89e-b60927b16593 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:56.841214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863a6fa-6337-4357-af18-d543238de994 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf67fa55f4d64b7ebeeb1490453661e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:56.842199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863a6fa-6337-4357-af18-d543238de994 service glance] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:56.842646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863a6fa-6337-4357-af18-d543238de994 service glance] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:56.843045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863a6fa-6337-4357-af18-d543238de994 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:56.857858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863a6fa-6337-4357-af18-d543238de994 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:56.857858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863a6fa-6337-4357-af18-d543238de994 service glance] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:56.872379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1095/2219] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:07:58.005668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d6cf7-37dc-4843-adc8-3c4feb73cbf6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:58.007944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c28d6cf7-37dc-4843-adc8-3c4feb73cbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:58.123144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:58.138051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d6cf7-37dc-4843-adc8-3c4feb73cbf6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:58.171923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:58.172980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:58.173335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:58.173684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] PATH_INFO: `/v3/users/331e4f7e78284c4bb0b05998d36c0f68` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:58.176837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] RBAC: Authorizing `identity:delete_user(user_id=331e4f7e78284c4bb0b05998d36c0f68)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:58.186152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:58.454968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331e4f7e78284c4bb0b05998d36c0f68'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:58.484807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331e4f7e78284c4bb0b05998d36c0f68'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:58.574319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331e4f7e78284c4bb0b05998d36c0f68'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:58.575091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331e4f7e78284c4bb0b05998d36c0f68'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:58.575765 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331e4f7e78284c4bb0b05998d36c0f68'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:07:58.601048 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1125/2220] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:07:58 2022] DELETE /identity/v3/users/331e4f7e78284c4bb0b05998d36c0f68 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:07:58.605567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea20f2e-94b5-4816-8f62-2830653803b5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:58.606079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea20f2e-94b5-4816-8f62-2830653803b5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:58.611888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea20f2e-94b5-4816-8f62-2830653803b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:58.656925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1ea20f2e-94b5-4816-8f62-2830653803b5 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:58.658165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1ea20f2e-94b5-4816-8f62-2830653803b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:58.832649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1096/2221] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:58.862290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:58.864085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:59.082183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb61ca3-a15d-4b54-a29a-37752c8d282a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:59.089318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc24d7-2fd3-46ac-b848-23e7a24f2baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:59.090374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc24d7-2fd3-46ac-b848-23e7a24f2baa service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:59.093227 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc24d7-2fd3-46ac-b848-23e7a24f2baa service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:59.093784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc24d7-2fd3-46ac-b848-23e7a24f2baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:59.114774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc24d7-2fd3-46ac-b848-23e7a24f2baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:59.116647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc24d7-2fd3-46ac-b848-23e7a24f2baa service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:59.128865 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1126/2222] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:59.471369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e05dbc-4401-420f-a1ba-7280541bc479 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:59.472116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e05dbc-4401-420f-a1ba-7280541bc479 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:59.472460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e05dbc-4401-420f-a1ba-7280541bc479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:59.481417 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:59.517258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-93e05dbc-4401-420f-a1ba-7280541bc479 None None] MFA Rules not processed for user `342a6239810e40aa83db81985af2bfc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:07:59.519591 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-93e05dbc-4401-420f-a1ba-7280541bc479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:59.606560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1097/2223] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:07:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:07:59.629903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc24d7-2fd3-46ac-b848-23e7a24f2baa service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:59.631375 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a1cc24d7-2fd3-46ac-b848-23e7a24f2baa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:59.746958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc24d7-2fd3-46ac-b848-23e7a24f2baa service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:59.751720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc55c6e-a2ad-439e-b1d1-bf17c073c167 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf67fa55f4d64b7ebeeb1490453661e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:59.752514 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc55c6e-a2ad-439e-b1d1-bf17c073c167 service glance] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:59.753031 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc55c6e-a2ad-439e-b1d1-bf17c073c167 service glance] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:59.753541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc55c6e-a2ad-439e-b1d1-bf17c073c167 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:59.760329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1cc55c6e-a2ad-439e-b1d1-bf17c073c167 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:59.807650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:07:59.849631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc55c6e-a2ad-439e-b1d1-bf17c073c167 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:07:59.851186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc55c6e-a2ad-439e-b1d1-bf17c073c167 service glance] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:07:59.864149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1127/2224] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:07:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:07:59.964213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e05dbc-4401-420f-a1ba-7280541bc479 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:07:59.965947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-93e05dbc-4401-420f-a1ba-7280541bc479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:07:59.984357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e05dbc-4401-420f-a1ba-7280541bc479 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:07:59.987413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:07:59.988480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:07:59.988931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:07:59.989677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] PATH_INFO: `/v3/projects/16721ae1330d4a03ac25d25b2ee8541b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:07:59.994103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16721ae1330d4a03ac25d25b2ee8541b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:00.003697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:00.167623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16721ae1330d4a03ac25d25b2ee8541b', 'request_id': 'req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:00.174799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16721ae1330d4a03ac25d25b2ee8541b', 'request_id': 'req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:00.194660 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff63e6-cb3d-41ff-8f22-3d6945748f73 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:00.195817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff63e6-cb3d-41ff-8f22-3d6945748f73 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:00.196109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff63e6-cb3d-41ff-8f22-3d6945748f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:00.214502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-41ff63e6-cb3d-41ff-8f22-3d6945748f73 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:00.215980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-41ff63e6-cb3d-41ff-8f22-3d6945748f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:00.250829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16721ae1330d4a03ac25d25b2ee8541b', 'request_id': 'req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:00.285897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] The token cache is being invalidate because project 16721ae1330d4a03ac25d25b2ee8541b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:00.286454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:00.287176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:00.292123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1098/2225] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:07:59 2022] DELETE /identity/v3/projects/16721ae1330d4a03ac25d25b2ee8541b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:00.293095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:00.294376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1128/2226] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:08:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:08:00.294617 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:00.386666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff63e6-cb3d-41ff-8f22-3d6945748f73 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:00.388133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-41ff63e6-cb3d-41ff-8f22-3d6945748f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:00.456616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bfd8b-6642-47c6-b9d0-52bf2f6fa152 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:00.471685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:00.472971 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:00.473460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:00.474088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] PATH_INFO: `/v3/users/ec06ecb3b57d45068cbab3cbbd0a48ce` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:00.475618 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec06ecb3b57d45068cbab3cbbd0a48ce)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:00.478578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:00.517014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff63e6-cb3d-41ff-8f22-3d6945748f73 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:00.523647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708126c-5c29-4f62-9e1f-8eff7c08bed0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:00.524506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708126c-5c29-4f62-9e1f-8eff7c08bed0 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:00.524788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708126c-5c29-4f62-9e1f-8eff7c08bed0 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:00.525066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708126c-5c29-4f62-9e1f-8eff7c08bed0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:00.533859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708126c-5c29-4f62-9e1f-8eff7c08bed0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:00.535493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708126c-5c29-4f62-9e1f-8eff7c08bed0 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:00.544717 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1129/2227] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:08:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:08:00.644312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:00.782514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec06ecb3b57d45068cbab3cbbd0a48ce'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:00.790998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec06ecb3b57d45068cbab3cbbd0a48ce'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:00.805573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec06ecb3b57d45068cbab3cbbd0a48ce'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:00.918371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec06ecb3b57d45068cbab3cbbd0a48ce'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:00.919084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ae159610-7a3e-4dc2-bc3f-deabdacfef3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec06ecb3b57d45068cbab3cbbd0a48ce'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:00.930982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1099/2228] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:00 2022] DELETE /identity/v3/users/ec06ecb3b57d45068cbab3cbbd0a48ce => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:01.601575 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708126c-5c29-4f62-9e1f-8eff7c08bed0 service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:01.621567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708126c-5c29-4f62-9e1f-8eff7c08bed0 service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:01.826337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:01.827278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:01.827524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:01.828562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] PATH_INFO: `/v3/projects/34c819496ff84af0a6fde5d7637be56c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:01.830496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c819496ff84af0a6fde5d7637be56c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:01.834057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:01.845124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:02.059378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c819496ff84af0a6fde5d7637be56c', 'request_id': 'req-4dc0f74b-063f-444e-8889-d7462ab5fdee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:02.074263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c819496ff84af0a6fde5d7637be56c', 'request_id': 'req-4dc0f74b-063f-444e-8889-d7462ab5fdee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:02.087371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c819496ff84af0a6fde5d7637be56c', 'request_id': 'req-4dc0f74b-063f-444e-8889-d7462ab5fdee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:02.266893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] The token cache is being invalidate because project 34c819496ff84af0a6fde5d7637be56c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:02.274423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:02.275525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:02.283882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1130/2229] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:01 2022] DELETE /identity/v3/projects/34c819496ff84af0a6fde5d7637be56c => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:02.289170 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99188eb-19cd-4759-b7bb-59046d50b428 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:02.289703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99188eb-19cd-4759-b7bb-59046d50b428 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:02.289703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99188eb-19cd-4759-b7bb-59046d50b428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:02.313354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a99188eb-19cd-4759-b7bb-59046d50b428 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:02.315756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a99188eb-19cd-4759-b7bb-59046d50b428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:02.390934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:02.392558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:02.466804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1100/2230] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:02.476739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16533-e7f7-4d53-8f1c-a173cb208bed None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:02.477617 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16533-e7f7-4d53-8f1c-a173cb208bed None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:02.477617 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16533-e7f7-4d53-8f1c-a173cb208bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:02.501696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-20b16533-e7f7-4d53-8f1c-a173cb208bed None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:02.505885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-20b16533-e7f7-4d53-8f1c-a173cb208bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:02.515248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1101/2231] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:02.520528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16533-e7f7-4d53-8f1c-a173cb208bed None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:02.538653 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16533-e7f7-4d53-8f1c-a173cb208bed None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:02.545186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ace42d-577b-4368-b1c3-800eaecb65b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:02.547093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ace42d-577b-4368-b1c3-800eaecb65b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:02.547877 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ace42d-577b-4368-b1c3-800eaecb65b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:02.548325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ace42d-577b-4368-b1c3-800eaecb65b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:02.550328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ace42d-577b-4368-b1c3-800eaecb65b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:08:02.557848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ace42d-577b-4368-b1c3-800eaecb65b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:02.557848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ace42d-577b-4368-b1c3-800eaecb65b0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:02.559863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0f74b-063f-444e-8889-d7462ab5fdee admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:02.572498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:02.573390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:02.573674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:02.573950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] PATH_INFO: `/v3/users/98613c1d81a7400e9dd21353e156e960` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:02.587993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=98613c1d81a7400e9dd21353e156e960)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:02.596201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:02.613949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1102/2232] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:08:02.618118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ace42d-577b-4368-b1c3-800eaecb65b0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:02.652605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ace42d-577b-4368-b1c3-800eaecb65b0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:02.661284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5a605-3fb1-40fc-9597-afa5e7f07392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:02.662333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5a605-3fb1-40fc-9597-afa5e7f07392 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:02.662791 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5a605-3fb1-40fc-9597-afa5e7f07392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:02.663396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5a605-3fb1-40fc-9597-afa5e7f07392 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:02.664580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5a605-3fb1-40fc-9597-afa5e7f07392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:02.669802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5a605-3fb1-40fc-9597-afa5e7f07392 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:02.776252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1103/2233] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:02 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:02.780452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5a605-3fb1-40fc-9597-afa5e7f07392 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:02.882800 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98613c1d81a7400e9dd21353e156e960'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:02.903476 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98613c1d81a7400e9dd21353e156e960'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:02.923315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:02.961173 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5a605-3fb1-40fc-9597-afa5e7f07392 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:02.964336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beb604-3554-4cd8-aafe-98e35d720c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:02.975977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beb604-3554-4cd8-aafe-98e35d720c3d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:02.976433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beb604-3554-4cd8-aafe-98e35d720c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:02.976873 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beb604-3554-4cd8-aafe-98e35d720c3d admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:02.977940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beb604-3554-4cd8-aafe-98e35d720c3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:02.979042 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98613c1d81a7400e9dd21353e156e960'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:02.979795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98613c1d81a7400e9dd21353e156e960'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:02.980516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995acb1e-0719-4616-be9f-5f783048bbcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98613c1d81a7400e9dd21353e156e960'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:02.980798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beb604-3554-4cd8-aafe-98e35d720c3d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:03.022870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1131/2234] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:02 2022] DELETE /identity/v3/users/98613c1d81a7400e9dd21353e156e960 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:03.024627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11181905-ea38-48f5-a5c6-a6f03272f326 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:03.028442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11181905-ea38-48f5-a5c6-a6f03272f326 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:03.028841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11181905-ea38-48f5-a5c6-a6f03272f326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:03.067429 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-11181905-ea38-48f5-a5c6-a6f03272f326 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:03.071774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-11181905-ea38-48f5-a5c6-a6f03272f326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:03.077500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1104/2235] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:02 2022] POST /identity/v3/users => generated 292 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:03.086745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beb604-3554-4cd8-aafe-98e35d720c3d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:03.097949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:03.107398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-39beb604-3554-4cd8-aafe-98e35d720c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:03.137458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beb604-3554-4cd8-aafe-98e35d720c3d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:03.168678 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1132/2236] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:03.169776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11181905-ea38-48f5-a5c6-a6f03272f326 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:03.171327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-11181905-ea38-48f5-a5c6-a6f03272f326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:03.240716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b611a9-b4eb-44ae-bc7f-5c7f178d76e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:03.241404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b611a9-b4eb-44ae-bc7f-5c7f178d76e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:03.241689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b611a9-b4eb-44ae-bc7f-5c7f178d76e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:03.241970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b611a9-b4eb-44ae-bc7f-5c7f178d76e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:03.242744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b611a9-b4eb-44ae-bc7f-5c7f178d76e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:03.244369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b611a9-b4eb-44ae-bc7f-5c7f178d76e5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:03.267029 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1105/2237] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:03.268569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b611a9-b4eb-44ae-bc7f-5c7f178d76e5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:03.296113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11181905-ea38-48f5-a5c6-a6f03272f326 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:03.304980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0669fe3-95ea-4899-adbf-935c2c9d87ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:03.305949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0669fe3-95ea-4899-adbf-935c2c9d87ee service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:03.306334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0669fe3-95ea-4899-adbf-935c2c9d87ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:03.306691 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0669fe3-95ea-4899-adbf-935c2c9d87ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:03.306825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b611a9-b4eb-44ae-bc7f-5c7f178d76e5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:03.320945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becc39f-35b5-46fb-8a3b-02ebf96d9df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:03.327890 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0669fe3-95ea-4899-adbf-935c2c9d87ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:03.328271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becc39f-35b5-46fb-8a3b-02ebf96d9df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:03.328271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becc39f-35b5-46fb-8a3b-02ebf96d9df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:03.328271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becc39f-35b5-46fb-8a3b-02ebf96d9df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:03.328631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0669fe3-95ea-4899-adbf-935c2c9d87ee service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:03.328712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becc39f-35b5-46fb-8a3b-02ebf96d9df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:03.328712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becc39f-35b5-46fb-8a3b-02ebf96d9df3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:03.341778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1133/2238] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:03.346652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0669fe3-95ea-4899-adbf-935c2c9d87ee service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:03.358515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c0669fe3-95ea-4899-adbf-935c2c9d87ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:03.363985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1106/2239] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:03.366539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becc39f-35b5-46fb-8a3b-02ebf96d9df3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:03.386010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becc39f-35b5-46fb-8a3b-02ebf96d9df3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:03.390483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0669fe3-95ea-4899-adbf-935c2c9d87ee service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:03.396776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:03.397482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:03.397766 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:03.398061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] PATH_INFO: `/v3/users/70b2afde553a455e841e440672584dac` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:03.399466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b2afde553a455e841e440672584dac)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:03.403615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:03.408726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12874f0-8b06-4849-ba28-0433724aa29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:03.411939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12874f0-8b06-4849-ba28-0433724aa29d admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:03.412560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12874f0-8b06-4849-ba28-0433724aa29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:03.416982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12874f0-8b06-4849-ba28-0433724aa29d admin admin] PATH_INFO: `/v3/projects/9bcb9ec4cda94d41915261ddee455455/users/84c6c84ee5fc409ba779660d2653c981/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:03.478976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12874f0-8b06-4849-ba28-0433724aa29d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bcb9ec4cda94d41915261ddee455455, user_id=84c6c84ee5fc409ba779660d2653c981, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:03.481882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12874f0-8b06-4849-ba28-0433724aa29d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:03.607573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1107/2240] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:03 2022] PUT /identity/v3/projects/9bcb9ec4cda94d41915261ddee455455/users/84c6c84ee5fc409ba779660d2653c981/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:03.619680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464596ad-5d75-49e9-b4c3-99701a4ecdc5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:03.620287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464596ad-5d75-49e9-b4c3-99701a4ecdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:03.620378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464596ad-5d75-49e9-b4c3-99701a4ecdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:03.668577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b2afde553a455e841e440672584dac'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:03.696020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b2afde553a455e841e440672584dac'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:03.697966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-464596ad-5d75-49e9-b4c3-99701a4ecdc5 None None] MFA Rules not processed for user `84c6c84ee5fc409ba779660d2653c981`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:03.699560 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-464596ad-5d75-49e9-b4c3-99701a4ecdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:03.806203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b2afde553a455e841e440672584dac'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:03.806959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b2afde553a455e841e440672584dac'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:03.807663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b2afde553a455e841e440672584dac'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:03.867528 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1134/2241] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:03 2022] DELETE /identity/v3/users/70b2afde553a455e841e440672584dac => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:03.882609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1108/2242] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:08:03.891687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:03.893393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:03.973329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:04.030195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779f392-18ef-4748-9a67-7520e93d7e57 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:04.036074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f87838-70c6-4b09-96bf-762d6761eb4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:04.036840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f87838-70c6-4b09-96bf-762d6761eb4e service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:04.037349 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f87838-70c6-4b09-96bf-762d6761eb4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:04.037774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f87838-70c6-4b09-96bf-762d6761eb4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:04.198644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f87838-70c6-4b09-96bf-762d6761eb4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:04.208000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f87838-70c6-4b09-96bf-762d6761eb4e service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:04.234306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1135/2243] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:04.428095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464596ad-5d75-49e9-b4c3-99701a4ecdc5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:04.432043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-464596ad-5d75-49e9-b4c3-99701a4ecdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:04.557526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:04.721264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464596ad-5d75-49e9-b4c3-99701a4ecdc5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:04.737085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f87838-70c6-4b09-96bf-762d6761eb4e service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:04.738920 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d7f87838-70c6-4b09-96bf-762d6761eb4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:04.973992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f87838-70c6-4b09-96bf-762d6761eb4e service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:04.977420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:04.978468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:04.981278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:04.981688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] PATH_INFO: `/v3/projects/b11e7ef755dd4b54a5dfe04db7249855` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:04.983573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b11e7ef755dd4b54a5dfe04db7249855)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:04.986119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:04.988324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:04.991637 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:04.992100 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:04.992545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] PATH_INFO: `/v3/projects/d766f73cac3f40bba6f961db2e005495` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:04.997468 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d766f73cac3f40bba6f961db2e005495)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:05.000366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:05.204806 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b11e7ef755dd4b54a5dfe04db7249855', 'request_id': 'req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.207480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd766f73cac3f40bba6f961db2e005495', 'request_id': 'req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.244768 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd766f73cac3f40bba6f961db2e005495', 'request_id': 'req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.248543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b11e7ef755dd4b54a5dfe04db7249855', 'request_id': 'req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.264193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd766f73cac3f40bba6f961db2e005495', 'request_id': 'req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.353001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b11e7ef755dd4b54a5dfe04db7249855', 'request_id': 'req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.375708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] The token cache is being invalidate because project b11e7ef755dd4b54a5dfe04db7249855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:05.376324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.381896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f5a65afb-e6e2-40ac-a42c-a11d405fbb28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.381896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1109/2244] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:04 2022] DELETE /identity/v3/projects/b11e7ef755dd4b54a5dfe04db7249855 => generated 0 bytes in 953 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:05.391667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c606-3a4a-4a3c-9cd2-20edcc42a763 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:05.392030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c606-3a4a-4a3c-9cd2-20edcc42a763 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:05.392407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c606-3a4a-4a3c-9cd2-20edcc42a763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:05.437908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] The token cache is being invalidate because project d766f73cac3f40bba6f961db2e005495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:05.438251 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.439926 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.445123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1136/2245] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:04 2022] DELETE /identity/v3/projects/d766f73cac3f40bba6f961db2e005495 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:05.445267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5588c606-3a4a-4a3c-9cd2-20edcc42a763 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:05.462737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:05.463333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5588c606-3a4a-4a3c-9cd2-20edcc42a763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:05.466438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:05.523829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:05.615747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1110/2246] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:05.626169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2e8e9-dfff-47c8-9cb1-47a83834ba4b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:05.626483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2e8e9-dfff-47c8-9cb1-47a83834ba4b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:05.626798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2e8e9-dfff-47c8-9cb1-47a83834ba4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:05.638247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99b3ee-4193-4c19-8fd1-b0ff4cdafec9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:05.645013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:05.645923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:05.646250 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:05.646526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] PATH_INFO: `/v3/users/f9dba100ffe24659b2b42e8617c3dc14` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:05.647581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:05.647819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9dba100ffe24659b2b42e8617c3dc14)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:05.649780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:05.660640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-23b2e8e9-dfff-47c8-9cb1-47a83834ba4b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:05.661874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-23b2e8e9-dfff-47c8-9cb1-47a83834ba4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:05.680989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1111/2247] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:08:05.684636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2e8e9-dfff-47c8-9cb1-47a83834ba4b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:05.696379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2e8e9-dfff-47c8-9cb1-47a83834ba4b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:05.701884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652726ff-2e32-4c5d-bac2-991e0b29e6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:05.702589 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652726ff-2e32-4c5d-bac2-991e0b29e6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:05.702870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652726ff-2e32-4c5d-bac2-991e0b29e6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:05.703153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652726ff-2e32-4c5d-bac2-991e0b29e6d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:05.703880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652726ff-2e32-4c5d-bac2-991e0b29e6d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:08:05.704182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652726ff-2e32-4c5d-bac2-991e0b29e6d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:05.706466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652726ff-2e32-4c5d-bac2-991e0b29e6d2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:05.723147 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1112/2248] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:08:05.727381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652726ff-2e32-4c5d-bac2-991e0b29e6d2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:05.739319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652726ff-2e32-4c5d-bac2-991e0b29e6d2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:05.749070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903ed94-744b-4f49-a0d7-4d3536ff57c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:05.749070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903ed94-744b-4f49-a0d7-4d3536ff57c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:05.749070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903ed94-744b-4f49-a0d7-4d3536ff57c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:05.749070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903ed94-744b-4f49-a0d7-4d3536ff57c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:05.749737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903ed94-744b-4f49-a0d7-4d3536ff57c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:05.750357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903ed94-744b-4f49-a0d7-4d3536ff57c4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:05.902888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1113/2249] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:05 2022] POST /identity/v3/projects => generated 399 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:08:05.918416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903ed94-744b-4f49-a0d7-4d3536ff57c4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:05.920255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9dba100ffe24659b2b42e8617c3dc14'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:05.933481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9dba100ffe24659b2b42e8617c3dc14'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:06.030073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903ed94-744b-4f49-a0d7-4d3536ff57c4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:06.036154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf6fa4-1b0f-4d88-a4a1-cf88849d116a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:06.036909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf6fa4-1b0f-4d88-a4a1-cf88849d116a admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:06.037303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf6fa4-1b0f-4d88-a4a1-cf88849d116a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:06.037512 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf6fa4-1b0f-4d88-a4a1-cf88849d116a admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:06.038578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf6fa4-1b0f-4d88-a4a1-cf88849d116a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:06.042772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf6fa4-1b0f-4d88-a4a1-cf88849d116a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:06.293104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9dba100ffe24659b2b42e8617c3dc14'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:06.293795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9dba100ffe24659b2b42e8617c3dc14'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:06.294504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9dba100ffe24659b2b42e8617c3dc14'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:06.315750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1137/2250] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:05 2022] DELETE /identity/v3/users/f9dba100ffe24659b2b42e8617c3dc14 => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:06.401914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1114/2251] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:05 2022] POST /identity/v3/users => generated 300 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:06.404190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:06.406637 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:06.553144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b5f78-78b2-485b-9d0b-afcdda2dcaab admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:06.564126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702f5d7-3fd4-4a7f-a6b7-0737a587f20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:06.566781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702f5d7-3fd4-4a7f-a6b7-0737a587f20f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:06.567169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702f5d7-3fd4-4a7f-a6b7-0737a587f20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:06.567538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702f5d7-3fd4-4a7f-a6b7-0737a587f20f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:06.568827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702f5d7-3fd4-4a7f-a6b7-0737a587f20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:06.571114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702f5d7-3fd4-4a7f-a6b7-0737a587f20f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:06.590787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1138/2252] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:06.595370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf6fa4-1b0f-4d88-a4a1-cf88849d116a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:06.607605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf6fa4-1b0f-4d88-a4a1-cf88849d116a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:06.626671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6c31-accc-455a-8626-8b433026fe92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:06.630325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6c31-accc-455a-8626-8b433026fe92 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:06.630325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6c31-accc-455a-8626-8b433026fe92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:06.630505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6c31-accc-455a-8626-8b433026fe92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:06.631215 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d6c31-accc-455a-8626-8b433026fe92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:06.632964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d6c31-accc-455a-8626-8b433026fe92 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:06.663016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702f5d7-3fd4-4a7f-a6b7-0737a587f20f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:06.675000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1115/2253] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:06.681830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702f5d7-3fd4-4a7f-a6b7-0737a587f20f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:06.687846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295e6df-06db-437c-b8a2-ef3ef91f4146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:06.688816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295e6df-06db-437c-b8a2-ef3ef91f4146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:06.689651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295e6df-06db-437c-b8a2-ef3ef91f4146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:06.689948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295e6df-06db-437c-b8a2-ef3ef91f4146 admin admin] PATH_INFO: `/v3/projects/95359cc27dc948f5a9fb45a128664b48/users/1382feca93a34d1c90dedc23c240c4a7/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:06.710930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295e6df-06db-437c-b8a2-ef3ef91f4146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95359cc27dc948f5a9fb45a128664b48, user_id=1382feca93a34d1c90dedc23c240c4a7, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:06.715956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295e6df-06db-437c-b8a2-ef3ef91f4146 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:06.774406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6c31-accc-455a-8626-8b433026fe92 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:06.776739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c93d6c31-accc-455a-8626-8b433026fe92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:06.802982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1139/2254] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:06 2022] PUT /identity/v3/projects/95359cc27dc948f5a9fb45a128664b48/users/1382feca93a34d1c90dedc23c240c4a7/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:06.816579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8681f78-4107-4694-9f73-7e3357b64f6e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:06.817740 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8681f78-4107-4694-9f73-7e3357b64f6e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:06.817868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8681f78-4107-4694-9f73-7e3357b64f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:06.822436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6c31-accc-455a-8626-8b433026fe92 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:06.842638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:06.867914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b8681f78-4107-4694-9f73-7e3357b64f6e None None] MFA Rules not processed for user `1382feca93a34d1c90dedc23c240c4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:06.869131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b8681f78-4107-4694-9f73-7e3357b64f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:06.945941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:06.946862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:06.947143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:06.947847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] PATH_INFO: `/v3/projects/f19246c2607b475bbb1dc5abd1ad2567` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:06.948972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f19246c2607b475bbb1dc5abd1ad2567)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:06.950946 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:06.962336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1140/2255] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:06.990504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8681f78-4107-4694-9f73-7e3357b64f6e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:06.998497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b8681f78-4107-4694-9f73-7e3357b64f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:07.092291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:07.097859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8681f78-4107-4694-9f73-7e3357b64f6e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:07.098013 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f19246c2607b475bbb1dc5abd1ad2567', 'request_id': 'req-179b1181-301e-4af4-a681-2092a40574aa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:07.110317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f19246c2607b475bbb1dc5abd1ad2567', 'request_id': 'req-179b1181-301e-4af4-a681-2092a40574aa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:07.177220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d380775-67c3-408c-86d3-87b20b62e038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:07.177912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d380775-67c3-408c-86d3-87b20b62e038 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:07.178240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d380775-67c3-408c-86d3-87b20b62e038 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:07.178557 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d380775-67c3-408c-86d3-87b20b62e038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:07.212869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f19246c2607b475bbb1dc5abd1ad2567', 'request_id': 'req-179b1181-301e-4af4-a681-2092a40574aa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:07.234369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] The token cache is being invalidate because project f19246c2607b475bbb1dc5abd1ad2567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:07.234831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:07.235759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-179b1181-301e-4af4-a681-2092a40574aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:07.238490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1116/2256] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:06 2022] DELETE /identity/v3/projects/f19246c2607b475bbb1dc5abd1ad2567 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:07.243771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc67fcb-fe3b-4be2-86a3-1b69de8a41bd None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:07.244186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc67fcb-fe3b-4be2-86a3-1b69de8a41bd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:07.244683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc67fcb-fe3b-4be2-86a3-1b69de8a41bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:07.265805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d380775-67c3-408c-86d3-87b20b62e038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:07.267771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d380775-67c3-408c-86d3-87b20b62e038 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:07.269273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3fc67fcb-fe3b-4be2-86a3-1b69de8a41bd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:07.270965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3fc67fcb-fe3b-4be2-86a3-1b69de8a41bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:07.278984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1141/2257] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:07.354252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1117/2258] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:08:07.363656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61972132-bf83-4ed4-99df-513eb7dcb1a0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:07.364226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61972132-bf83-4ed4-99df-513eb7dcb1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:07.364226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61972132-bf83-4ed4-99df-513eb7dcb1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:07.386285 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:07.395432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-61972132-bf83-4ed4-99df-513eb7dcb1a0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:07.396772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-61972132-bf83-4ed4-99df-513eb7dcb1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:07.413121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1142/2259] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:07.419449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc67fcb-fe3b-4be2-86a3-1b69de8a41bd None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:07.430814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc67fcb-fe3b-4be2-86a3-1b69de8a41bd None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:07.440957 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52537de-c905-4acb-936e-8b221bc2ac8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:07.442183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52537de-c905-4acb-936e-8b221bc2ac8c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:07.443021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52537de-c905-4acb-936e-8b221bc2ac8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:07.452753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52537de-c905-4acb-936e-8b221bc2ac8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:07.453674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52537de-c905-4acb-936e-8b221bc2ac8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:08:07.454060 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52537de-c905-4acb-936e-8b221bc2ac8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:07.467267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52537de-c905-4acb-936e-8b221bc2ac8c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:07.491582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1118/2260] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:08:07.492715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61972132-bf83-4ed4-99df-513eb7dcb1a0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:07.508718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61972132-bf83-4ed4-99df-513eb7dcb1a0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:07.512026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c1a7e-9694-4348-b0b6-a793edcc080a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:07.512706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c1a7e-9694-4348-b0b6-a793edcc080a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:07.513200 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c1a7e-9694-4348-b0b6-a793edcc080a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:07.513482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c1a7e-9694-4348-b0b6-a793edcc080a admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:07.514527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c1a7e-9694-4348-b0b6-a793edcc080a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:07.516449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c1a7e-9694-4348-b0b6-a793edcc080a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:07.655342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1143/2261] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:07 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:07.655870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52537de-c905-4acb-936e-8b221bc2ac8c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:07.671402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52537de-c905-4acb-936e-8b221bc2ac8c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:07.742273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6deb587-1f83-4872-8ed3-2f6c171f36f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:07.743270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6deb587-1f83-4872-8ed3-2f6c171f36f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:07.743722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6deb587-1f83-4872-8ed3-2f6c171f36f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:07.744098 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6deb587-1f83-4872-8ed3-2f6c171f36f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:07.752286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6deb587-1f83-4872-8ed3-2f6c171f36f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:07.754928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6deb587-1f83-4872-8ed3-2f6c171f36f0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:07.880431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1119/2262] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:07 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:07.881748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c1a7e-9694-4348-b0b6-a793edcc080a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:07.897508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c1a7e-9694-4348-b0b6-a793edcc080a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:07.910900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc827ceb-f379-4288-87d6-0316c2e5432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:07.911979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc827ceb-f379-4288-87d6-0316c2e5432a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:07.914347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc827ceb-f379-4288-87d6-0316c2e5432a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:07.914630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc827ceb-f379-4288-87d6-0316c2e5432a admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:07.915432 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc827ceb-f379-4288-87d6-0316c2e5432a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:07.917128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc827ceb-f379-4288-87d6-0316c2e5432a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:07.934987 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1144/2263] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:07.939148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6deb587-1f83-4872-8ed3-2f6c171f36f0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:07.953575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6deb587-1f83-4872-8ed3-2f6c171f36f0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:07.964558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77e6a6-1936-4219-a364-6954fc14309f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:07.965249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77e6a6-1936-4219-a364-6954fc14309f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:07.965575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77e6a6-1936-4219-a364-6954fc14309f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:07.965859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77e6a6-1936-4219-a364-6954fc14309f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:07.977401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77e6a6-1936-4219-a364-6954fc14309f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:07.982287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77e6a6-1936-4219-a364-6954fc14309f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:07.998179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1120/2264] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:07.999413 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc827ceb-f379-4288-87d6-0316c2e5432a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:08.015417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc827ceb-f379-4288-87d6-0316c2e5432a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:08.022350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412df44-6ba7-4792-8730-f152ed5fc7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:08.026689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412df44-6ba7-4792-8730-f152ed5fc7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:08.026689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412df44-6ba7-4792-8730-f152ed5fc7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:08.026689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412df44-6ba7-4792-8730-f152ed5fc7bc admin admin] PATH_INFO: `/v3/projects/ca67a2ecd29f4e0dae6e4a2e64296d51/users/aada5d1fdafd41d390d58abd5ebb5481/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:08.048282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412df44-6ba7-4792-8730-f152ed5fc7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca67a2ecd29f4e0dae6e4a2e64296d51, user_id=aada5d1fdafd41d390d58abd5ebb5481, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:08.051681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412df44-6ba7-4792-8730-f152ed5fc7bc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:08.162327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1145/2265] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:07 2022] PUT /identity/v3/projects/ca67a2ecd29f4e0dae6e4a2e64296d51/users/aada5d1fdafd41d390d58abd5ebb5481/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:08:08.163190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af57a-66d5-4b38-b6b7-5a64dd1b7ee9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:08.163488 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af57a-66d5-4b38-b6b7-5a64dd1b7ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:08.163775 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af57a-66d5-4b38-b6b7-5a64dd1b7ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:08.202785 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-096af57a-66d5-4b38-b6b7-5a64dd1b7ee9 None None] MFA Rules not processed for user `aada5d1fdafd41d390d58abd5ebb5481`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:08.204358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-096af57a-66d5-4b38-b6b7-5a64dd1b7ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:08.261709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:08.374490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1121/2266] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:08:08.410381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412df44-6ba7-4792-8730-f152ed5fc7bc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:08.415884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2412df44-6ba7-4792-8730-f152ed5fc7bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:08.463299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096af57a-66d5-4b38-b6b7-5a64dd1b7ee9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:08.463299 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-096af57a-66d5-4b38-b6b7-5a64dd1b7ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:08.599518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096af57a-66d5-4b38-b6b7-5a64dd1b7ee9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:08.604129 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abc457-5c2e-4bb8-9a4f-9aa7155d9836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:08.604869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abc457-5c2e-4bb8-9a4f-9aa7155d9836 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:08.605202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abc457-5c2e-4bb8-9a4f-9aa7155d9836 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:08.605435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abc457-5c2e-4bb8-9a4f-9aa7155d9836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:08.615712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abc457-5c2e-4bb8-9a4f-9aa7155d9836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:08.617444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abc457-5c2e-4bb8-9a4f-9aa7155d9836 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:08.629457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412df44-6ba7-4792-8730-f152ed5fc7bc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:08.644179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1122/2267] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:08:08.649696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:08.650653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:08.650932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:08.651228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] PATH_INFO: `/v3/users/84c6c84ee5fc409ba779660d2653c981` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:08.658142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abc457-5c2e-4bb8-9a4f-9aa7155d9836 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:08.658993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c6c84ee5fc409ba779660d2653c981)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:08.671001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abc457-5c2e-4bb8-9a4f-9aa7155d9836 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:08.672075 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:08.680937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:08.682130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:08.682499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:08.682848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] PATH_INFO: `/v3/users/aada5d1fdafd41d390d58abd5ebb5481` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:08.684212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aada5d1fdafd41d390d58abd5ebb5481)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:08.686385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:08.991268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aada5d1fdafd41d390d58abd5ebb5481'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:08.999210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aada5d1fdafd41d390d58abd5ebb5481'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:09.000592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c6c84ee5fc409ba779660d2653c981'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:09.021403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c6c84ee5fc409ba779660d2653c981'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:09.025035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aada5d1fdafd41d390d58abd5ebb5481'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:09.098742 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c6c84ee5fc409ba779660d2653c981'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:09.099535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c6c84ee5fc409ba779660d2653c981'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:09.100231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6580013c-b3b5-481d-82f6-a22fa39b2d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c6c84ee5fc409ba779660d2653c981'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:09.123427 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1146/2268] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:08 2022] DELETE /identity/v3/users/84c6c84ee5fc409ba779660d2653c981 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:09.128590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68439a-f221-4a16-8b8e-9b26d6ffb580 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:09.128882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68439a-f221-4a16-8b8e-9b26d6ffb580 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:09.129159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68439a-f221-4a16-8b8e-9b26d6ffb580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:09.159022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3a68439a-f221-4a16-8b8e-9b26d6ffb580 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:09.162860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3a68439a-f221-4a16-8b8e-9b26d6ffb580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:09.167758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aada5d1fdafd41d390d58abd5ebb5481'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:09.171835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-655245c1-8f4a-42e7-95f2-044d61354180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aada5d1fdafd41d390d58abd5ebb5481'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:09.176388 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1123/2269] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:08 2022] DELETE /identity/v3/users/aada5d1fdafd41d390d58abd5ebb5481 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:09.180966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59c7a-7882-4e21-a00b-202a36bac6c5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:09.181310 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59c7a-7882-4e21-a00b-202a36bac6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:09.181600 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59c7a-7882-4e21-a00b-202a36bac6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:09.188155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:09.202300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f5e59c7a-7882-4e21-a00b-202a36bac6c5 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:09.203325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f5e59c7a-7882-4e21-a00b-202a36bac6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:09.270080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1147/2270] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:08:09.277384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68439a-f221-4a16-8b8e-9b26d6ffb580 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:09.278788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3a68439a-f221-4a16-8b8e-9b26d6ffb580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:09.279264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:09.343122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1124/2271] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:09.352254 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59c7a-7882-4e21-a00b-202a36bac6c5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:09.353965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f5e59c7a-7882-4e21-a00b-202a36bac6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:09.416750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68439a-f221-4a16-8b8e-9b26d6ffb580 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:09.461570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59c7a-7882-4e21-a00b-202a36bac6c5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:09.465748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9baac-0c1f-47b4-bcef-8687d0e4a047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:09.466554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9baac-0c1f-47b4-bcef-8687d0e4a047 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:09.467043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9baac-0c1f-47b4-bcef-8687d0e4a047 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:09.467585 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9baac-0c1f-47b4-bcef-8687d0e4a047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:09.480566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9baac-0c1f-47b4-bcef-8687d0e4a047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:09.482598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9baac-0c1f-47b4-bcef-8687d0e4a047 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:09.534615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1125/2272] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:08:09.558274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93237914-af21-4bcf-b8a9-d1782590c3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:09.559041 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93237914-af21-4bcf-b8a9-d1782590c3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:09.559405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93237914-af21-4bcf-b8a9-d1782590c3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:09.559764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93237914-af21-4bcf-b8a9-d1782590c3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:09.576829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93237914-af21-4bcf-b8a9-d1782590c3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:09.602614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93237914-af21-4bcf-b8a9-d1782590c3d1 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:09.641319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1148/2273] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:08:11.271459 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9baac-0c1f-47b4-bcef-8687d0e4a047 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:11.273006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a7d9baac-0c1f-47b4-bcef-8687d0e4a047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:11.291683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9baac-0c1f-47b4-bcef-8687d0e4a047 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:11.294926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:11.295792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:11.296068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:11.296335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] PATH_INFO: `/v3/projects/9bcb9ec4cda94d41915261ddee455455` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:11.297598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bcb9ec4cda94d41915261ddee455455)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:11.300527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:11.319515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93237914-af21-4bcf-b8a9-d1782590c3d1 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:11.320109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-93237914-af21-4bcf-b8a9-d1782590c3d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:11.349308 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93237914-af21-4bcf-b8a9-d1782590c3d1 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:11.351860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:11.352806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:11.353152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:11.353508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] PATH_INFO: `/v3/projects/ca67a2ecd29f4e0dae6e4a2e64296d51` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:11.354788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca67a2ecd29f4e0dae6e4a2e64296d51)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:11.357114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:11.426752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bcb9ec4cda94d41915261ddee455455', 'request_id': 'req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:11.433409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bcb9ec4cda94d41915261ddee455455', 'request_id': 'req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:11.485307 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca67a2ecd29f4e0dae6e4a2e64296d51', 'request_id': 'req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:11.492415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca67a2ecd29f4e0dae6e4a2e64296d51', 'request_id': 'req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:11.499844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca67a2ecd29f4e0dae6e4a2e64296d51', 'request_id': 'req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:11.567232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bcb9ec4cda94d41915261ddee455455', 'request_id': 'req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:11.581224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] The token cache is being invalidate because project 9bcb9ec4cda94d41915261ddee455455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:11.581224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:11.581816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a55d5c6-4784-4ae4-bc39-d0868b3e1f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:11.583509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1126/2274] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:11 2022] DELETE /identity/v3/projects/9bcb9ec4cda94d41915261ddee455455 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:08:11.588519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbf97d-970f-445d-8d26-1b39f628832a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:11.594911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbf97d-970f-445d-8d26-1b39f628832a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:11.594911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbf97d-970f-445d-8d26-1b39f628832a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:11.631255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-62dbf97d-970f-445d-8d26-1b39f628832a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:11.632506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-62dbf97d-970f-445d-8d26-1b39f628832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:11.707512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] The token cache is being invalidate because project ca67a2ecd29f4e0dae6e4a2e64296d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:11.707512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:11.710503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9b70daa0-f9bc-4c79-8c68-d85ebd18dc5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:11.719782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1149/2275] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:11 2022] DELETE /identity/v3/projects/ca67a2ecd29f4e0dae6e4a2e64296d51 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:08:11.724072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aadb1-d56c-4d69-8775-7a08e93daa16 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:11.724363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aadb1-d56c-4d69-8775-7a08e93daa16 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:11.724644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aadb1-d56c-4d69-8775-7a08e93daa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:11.744516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:11.770999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-720aadb1-d56c-4d69-8775-7a08e93daa16 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:11.772712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-720aadb1-d56c-4d69-8775-7a08e93daa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:11.797869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1127/2276] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:11.807009 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d40c2-7cec-4536-bab3-7def61e1301b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:11.807334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d40c2-7cec-4536-bab3-7def61e1301b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:11.810324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d40c2-7cec-4536-bab3-7def61e1301b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:11.840737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5b6d40c2-7cec-4536-bab3-7def61e1301b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:11.842307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5b6d40c2-7cec-4536-bab3-7def61e1301b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:11.852588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1128/2277] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:11.870014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d40c2-7cec-4536-bab3-7def61e1301b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:11.878556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:11.885671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d40c2-7cec-4536-bab3-7def61e1301b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:11.891780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fa508-90fa-4254-9a48-97dedcf1e50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:11.892553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fa508-90fa-4254-9a48-97dedcf1e50f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:11.892903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fa508-90fa-4254-9a48-97dedcf1e50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:11.893250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fa508-90fa-4254-9a48-97dedcf1e50f admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:11.894158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fa508-90fa-4254-9a48-97dedcf1e50f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:08:11.894554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fa508-90fa-4254-9a48-97dedcf1e50f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:11.896262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fa508-90fa-4254-9a48-97dedcf1e50f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:11.899219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1150/2278] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:11.907355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed325d5-0a7b-438a-a040-b3d42793ed2c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:11.908521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed325d5-0a7b-438a-a040-b3d42793ed2c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:11.909078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed325d5-0a7b-438a-a040-b3d42793ed2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:11.930317 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1129/2279] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:08:11.931416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fa508-90fa-4254-9a48-97dedcf1e50f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:11.932174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2ed325d5-0a7b-438a-a040-b3d42793ed2c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:11.933387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2ed325d5-0a7b-438a-a040-b3d42793ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:11.942774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fa508-90fa-4254-9a48-97dedcf1e50f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:11.945979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1151/2280] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:08:11.949963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed325d5-0a7b-438a-a040-b3d42793ed2c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:11.951381 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a512905-4250-4585-915f-781960c52043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:11.952113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512905-4250-4585-915f-781960c52043 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:11.952460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512905-4250-4585-915f-781960c52043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:11.952808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512905-4250-4585-915f-781960c52043 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:11.953565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a512905-4250-4585-915f-781960c52043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:11.956734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a512905-4250-4585-915f-781960c52043 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:11.960314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed325d5-0a7b-438a-a040-b3d42793ed2c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:11.968148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd08ed3-ba21-4bac-a659-dc66a141e98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:11.968881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd08ed3-ba21-4bac-a659-dc66a141e98c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:11.969145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd08ed3-ba21-4bac-a659-dc66a141e98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:11.969417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd08ed3-ba21-4bac-a659-dc66a141e98c admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:11.970344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd08ed3-ba21-4bac-a659-dc66a141e98c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:08:11.970699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd08ed3-ba21-4bac-a659-dc66a141e98c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:11.972472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd08ed3-ba21-4bac-a659-dc66a141e98c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:11.986110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1152/2281] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:08:11.989086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd08ed3-ba21-4bac-a659-dc66a141e98c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.000581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd08ed3-ba21-4bac-a659-dc66a141e98c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.006283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b9a9f-0151-4e18-b2a9-50b70b665b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.007684 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b9a9f-0151-4e18-b2a9-50b70b665b41 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.009332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b9a9f-0151-4e18-b2a9-50b70b665b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.009689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b9a9f-0151-4e18-b2a9-50b70b665b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.010507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b9a9f-0151-4e18-b2a9-50b70b665b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.012683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b9a9f-0151-4e18-b2a9-50b70b665b41 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.028836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1130/2282] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:11 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:08:12.033264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a512905-4250-4585-915f-781960c52043 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.107186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a512905-4250-4585-915f-781960c52043 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.114866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f145-54dc-4fc5-aa1a-86700d557347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.115550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f145-54dc-4fc5-aa1a-86700d557347 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.115837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f145-54dc-4fc5-aa1a-86700d557347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.116113 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f145-54dc-4fc5-aa1a-86700d557347 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.116818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08f145-54dc-4fc5-aa1a-86700d557347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.118814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08f145-54dc-4fc5-aa1a-86700d557347 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.146598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1153/2283] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:11 2022] POST /identity/v3/projects => generated 389 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:08:12.152851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b9a9f-0151-4e18-b2a9-50b70b665b41 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.200278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1131/2284] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:12 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:12.206689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f145-54dc-4fc5-aa1a-86700d557347 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.237672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b9a9f-0151-4e18-b2a9-50b70b665b41 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.244975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f21a2-605d-4e8c-bc87-daaccd25e494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.245932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f21a2-605d-4e8c-bc87-daaccd25e494 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.247029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f21a2-605d-4e8c-bc87-daaccd25e494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.247677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f21a2-605d-4e8c-bc87-daaccd25e494 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.248531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f21a2-605d-4e8c-bc87-daaccd25e494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.252822 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f21a2-605d-4e8c-bc87-daaccd25e494 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.258897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f145-54dc-4fc5-aa1a-86700d557347 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.276278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdcbec-bc2b-4408-818c-4bd32e05929c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.279293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdcbec-bc2b-4408-818c-4bd32e05929c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.279293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdcbec-bc2b-4408-818c-4bd32e05929c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.279293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdcbec-bc2b-4408-818c-4bd32e05929c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.280476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdcbec-bc2b-4408-818c-4bd32e05929c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.282268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdcbec-bc2b-4408-818c-4bd32e05929c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.298886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1132/2285] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:12.300698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdcbec-bc2b-4408-818c-4bd32e05929c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.312405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdcbec-bc2b-4408-818c-4bd32e05929c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.323419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9cd51-f62e-4b9b-8689-ce8e6b8be70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.324103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cd51-f62e-4b9b-8689-ce8e6b8be70c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.324390 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cd51-f62e-4b9b-8689-ce8e6b8be70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.324669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cd51-f62e-4b9b-8689-ce8e6b8be70c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.325452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9cd51-f62e-4b9b-8689-ce8e6b8be70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.327609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9cd51-f62e-4b9b-8689-ce8e6b8be70c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.345675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1133/2286] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:12.352190 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9cd51-f62e-4b9b-8689-ce8e6b8be70c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.365213 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1154/2287] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:12 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:12.367904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9cd51-f62e-4b9b-8689-ce8e6b8be70c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.368597 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f21a2-605d-4e8c-bc87-daaccd25e494 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.373005 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3cb7a-4fc7-40e1-8e81-34babbad59c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.377088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3cb7a-4fc7-40e1-8e81-34babbad59c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.377088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3cb7a-4fc7-40e1-8e81-34babbad59c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.377088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3cb7a-4fc7-40e1-8e81-34babbad59c1 admin admin] PATH_INFO: `/v3/projects/84c9eff020fd4de4b2a3c1528b668485/users/3be4829f03804026b9543247abc7b82f/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.383952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f21a2-605d-4e8c-bc87-daaccd25e494 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.393119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3cb7a-4fc7-40e1-8e81-34babbad59c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c9eff020fd4de4b2a3c1528b668485, user_id=3be4829f03804026b9543247abc7b82f, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.396268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0b629-9f74-4597-8a0e-b3538c5e8c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.396428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3cb7a-4fc7-40e1-8e81-34babbad59c1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.396976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0b629-9f74-4597-8a0e-b3538c5e8c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.397265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0b629-9f74-4597-8a0e-b3538c5e8c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.397530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0b629-9f74-4597-8a0e-b3538c5e8c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.398317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0b629-9f74-4597-8a0e-b3538c5e8c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.400033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0b629-9f74-4597-8a0e-b3538c5e8c7b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.416239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1155/2288] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:12.419836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0b629-9f74-4597-8a0e-b3538c5e8c7b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.431327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0b629-9f74-4597-8a0e-b3538c5e8c7b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.436580 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e96ef-764b-438b-8d7e-1f387d2a0ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.437252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e96ef-764b-438b-8d7e-1f387d2a0ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.437537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e96ef-764b-438b-8d7e-1f387d2a0ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.437813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e96ef-764b-438b-8d7e-1f387d2a0ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.438606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e96ef-764b-438b-8d7e-1f387d2a0ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.441201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e96ef-764b-438b-8d7e-1f387d2a0ec2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.454078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1156/2289] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:08:12.457636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e96ef-764b-438b-8d7e-1f387d2a0ec2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.467889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e96ef-764b-438b-8d7e-1f387d2a0ec2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.472952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c1315-b84f-4f7d-8d0c-1b2da36b4c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.473844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c1315-b84f-4f7d-8d0c-1b2da36b4c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.474290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c1315-b84f-4f7d-8d0c-1b2da36b4c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.479888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c1315-b84f-4f7d-8d0c-1b2da36b4c2e admin admin] PATH_INFO: `/v3/projects/cf31b51aef494e8d9893321735325126/users/fd34cb0098f44ee0afa9c64857ea82da/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.484030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1134/2290] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:12 2022] PUT /identity/v3/projects/84c9eff020fd4de4b2a3c1528b668485/users/3be4829f03804026b9543247abc7b82f/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:12.495983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2628fa-3c12-4ffe-b5ac-ee9f5ec18f3b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.496476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2628fa-3c12-4ffe-b5ac-ee9f5ec18f3b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.496766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2628fa-3c12-4ffe-b5ac-ee9f5ec18f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.502538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c1315-b84f-4f7d-8d0c-1b2da36b4c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf31b51aef494e8d9893321735325126, user_id=fd34cb0098f44ee0afa9c64857ea82da, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.508653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c1315-b84f-4f7d-8d0c-1b2da36b4c2e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.529047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8b2628fa-3c12-4ffe-b5ac-ee9f5ec18f3b None None] MFA Rules not processed for user `3be4829f03804026b9543247abc7b82f`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:12.531678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8b2628fa-3c12-4ffe-b5ac-ee9f5ec18f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:12.606826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1157/2291] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:12 2022] PUT /identity/v3/projects/cf31b51aef494e8d9893321735325126/users/fd34cb0098f44ee0afa9c64857ea82da/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:08:12.619564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d94b53-8a54-4259-a73f-613874882f04 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.619564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d94b53-8a54-4259-a73f-613874882f04 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.620531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d94b53-8a54-4259-a73f-613874882f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.622722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1135/2292] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:12.633801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:12.637713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2628fa-3c12-4ffe-b5ac-ee9f5ec18f3b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.639664 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8b2628fa-3c12-4ffe-b5ac-ee9f5ec18f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:12.648657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:12.654244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c4d94b53-8a54-4259-a73f-613874882f04 None None] MFA Rules not processed for user `fd34cb0098f44ee0afa9c64857ea82da`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:12.655814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c4d94b53-8a54-4259-a73f-613874882f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:12.759241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2628fa-3c12-4ffe-b5ac-ee9f5ec18f3b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.778226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1158/2293] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:12.824593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f87d87-4561-4c29-aaa5-f04d3e9f3d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.825637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f87d87-4561-4c29-aaa5-f04d3e9f3d18 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.825919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f87d87-4561-4c29-aaa5-f04d3e9f3d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.826215 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f87d87-4561-4c29-aaa5-f04d3e9f3d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.842983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d94b53-8a54-4259-a73f-613874882f04 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:12.844066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c4d94b53-8a54-4259-a73f-613874882f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:12.857360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d94b53-8a54-4259-a73f-613874882f04 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:12.861718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dee28-f5a5-4062-b497-ba7929900907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:12.862596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dee28-f5a5-4062-b497-ba7929900907 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:12.862901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dee28-f5a5-4062-b497-ba7929900907 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:12.863157 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dee28-f5a5-4062-b497-ba7929900907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:12.873663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dee28-f5a5-4062-b497-ba7929900907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.875029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dee28-f5a5-4062-b497-ba7929900907 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.885503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1159/2294] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:08:12.916835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f87d87-4561-4c29-aaa5-f04d3e9f3d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:12.918735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f87d87-4561-4c29-aaa5-f04d3e9f3d18 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:12.932668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1136/2295] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:08:12.960443 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dee28-f5a5-4062-b497-ba7929900907 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:13.071433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dee28-f5a5-4062-b497-ba7929900907 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:13.078066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:13.078894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:13.079184 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:13.079508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] PATH_INFO: `/v3/users/3be4829f03804026b9543247abc7b82f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:13.081363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be4829f03804026b9543247abc7b82f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:13.083346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:13.319340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be4829f03804026b9543247abc7b82f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:13.331201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be4829f03804026b9543247abc7b82f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:13.461268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be4829f03804026b9543247abc7b82f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:13.462400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be4829f03804026b9543247abc7b82f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:13.463496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be4829f03804026b9543247abc7b82f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:13.492320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1160/2296] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:12 2022] DELETE /identity/v3/users/3be4829f03804026b9543247abc7b82f => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:13.497495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1a118-b461-4578-8cbf-226875607867 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:13.497495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1a118-b461-4578-8cbf-226875607867 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:13.497495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1a118-b461-4578-8cbf-226875607867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:13.517538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-85c1a118-b461-4578-8cbf-226875607867 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:13.519509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-85c1a118-b461-4578-8cbf-226875607867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:13.641057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1137/2297] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:13.651609 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:13.653563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:13.659273 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:13.739642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d53e8c-8971-48db-a94b-405d47121216 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:13.744226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3dca0-8024-4ed9-b66e-2d865c9d7ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:13.745439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3dca0-8024-4ed9-b66e-2d865c9d7ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:13.745727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3dca0-8024-4ed9-b66e-2d865c9d7ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:13.746004 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3dca0-8024-4ed9-b66e-2d865c9d7ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:13.759755 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3dca0-8024-4ed9-b66e-2d865c9d7ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:13.761430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3dca0-8024-4ed9-b66e-2d865c9d7ad7 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:13.797573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1161/2298] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:14.294937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1a118-b461-4578-8cbf-226875607867 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:14.300494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-85c1a118-b461-4578-8cbf-226875607867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:14.317945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1a118-b461-4578-8cbf-226875607867 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:14.329255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:14.329255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:14.329255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:14.329255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] PATH_INFO: `/v3/projects/84c9eff020fd4de4b2a3c1528b668485` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:14.354627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c9eff020fd4de4b2a3c1528b668485)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:14.354627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:14.354627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:14.479968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c9eff020fd4de4b2a3c1528b668485', 'request_id': 'req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:14.489316 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c9eff020fd4de4b2a3c1528b668485', 'request_id': 'req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:14.630221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c9eff020fd4de4b2a3c1528b668485', 'request_id': 'req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:14.657971 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] The token cache is being invalidate because project 84c9eff020fd4de4b2a3c1528b668485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:14.676476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:14.676476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ea9a1f4c-d65d-4699-89b0-8acdf7d9cde9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:14.676476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1138/2299] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:14 2022] DELETE /identity/v3/projects/84c9eff020fd4de4b2a3c1528b668485 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:08:14.680923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0207a-8822-48da-aa6f-71ffb771ac69 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:14.681451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0207a-8822-48da-aa6f-71ffb771ac69 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:14.681559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0207a-8822-48da-aa6f-71ffb771ac69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:14.707681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1dd0207a-8822-48da-aa6f-71ffb771ac69 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:14.708940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1dd0207a-8822-48da-aa6f-71ffb771ac69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:14.794384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1162/2300] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:14.802566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e77c9-b57e-4e9f-9a84-85d7dafeae2a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:14.802566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e77c9-b57e-4e9f-9a84-85d7dafeae2a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:14.802862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e77c9-b57e-4e9f-9a84-85d7dafeae2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:14.833982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-906e77c9-b57e-4e9f-9a84-85d7dafeae2a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:14.835287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-906e77c9-b57e-4e9f-9a84-85d7dafeae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:14.846067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1139/2301] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:08:14.851373 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0207a-8822-48da-aa6f-71ffb771ac69 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:14.881204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0207a-8822-48da-aa6f-71ffb771ac69 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:14.886003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa99c4-4b4f-4a7d-a953-2c6ab6e7a2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:14.886703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa99c4-4b4f-4a7d-a953-2c6ab6e7a2fc admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:14.886986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa99c4-4b4f-4a7d-a953-2c6ab6e7a2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:14.887263 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa99c4-4b4f-4a7d-a953-2c6ab6e7a2fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:14.887990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa99c4-4b4f-4a7d-a953-2c6ab6e7a2fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:08:14.888298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa99c4-4b4f-4a7d-a953-2c6ab6e7a2fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:14.890175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa99c4-4b4f-4a7d-a953-2c6ab6e7a2fc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:14.906520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1163/2302] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:08:14.908999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e77c9-b57e-4e9f-9a84-85d7dafeae2a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:14.921514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e77c9-b57e-4e9f-9a84-85d7dafeae2a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:14.927650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64491fdb-ee9e-422d-bcc7-f41d867c8616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:14.928375 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64491fdb-ee9e-422d-bcc7-f41d867c8616 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:14.928738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64491fdb-ee9e-422d-bcc7-f41d867c8616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:14.929074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64491fdb-ee9e-422d-bcc7-f41d867c8616 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:14.929870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64491fdb-ee9e-422d-bcc7-f41d867c8616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:14.931846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64491fdb-ee9e-422d-bcc7-f41d867c8616 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:15.261855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1140/2303] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:14 2022] POST /identity/v3/projects => generated 371 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:08:15.266392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa99c4-4b4f-4a7d-a953-2c6ab6e7a2fc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:15.277358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa99c4-4b4f-4a7d-a953-2c6ab6e7a2fc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:15.359564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5c765-f2d0-4dff-bb68-d9abb5cb16e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:15.361590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5c765-f2d0-4dff-bb68-d9abb5cb16e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:15.361590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5c765-f2d0-4dff-bb68-d9abb5cb16e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:15.361590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5c765-f2d0-4dff-bb68-d9abb5cb16e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:15.362850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5c765-f2d0-4dff-bb68-d9abb5cb16e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:15.364871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5c765-f2d0-4dff-bb68-d9abb5cb16e8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:15.499008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1164/2304] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:15 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:15.513258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64491fdb-ee9e-422d-bcc7-f41d867c8616 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:15.524529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64491fdb-ee9e-422d-bcc7-f41d867c8616 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:15.531991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2c098-e11d-44b6-b23b-ce86d5a9ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:15.532719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2c098-e11d-44b6-b23b-ce86d5a9ca06 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:15.532999 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2c098-e11d-44b6-b23b-ce86d5a9ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:15.533310 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2c098-e11d-44b6-b23b-ce86d5a9ca06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:15.534133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2c098-e11d-44b6-b23b-ce86d5a9ca06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:15.535720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2c098-e11d-44b6-b23b-ce86d5a9ca06 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:15.551259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1141/2305] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:15.555216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5c765-f2d0-4dff-bb68-d9abb5cb16e8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:15.567674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5c765-f2d0-4dff-bb68-d9abb5cb16e8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:15.572772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8eb70-0f21-4220-a335-e7493ee90c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:15.573439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8eb70-0f21-4220-a335-e7493ee90c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:15.573721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8eb70-0f21-4220-a335-e7493ee90c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:15.573997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8eb70-0f21-4220-a335-e7493ee90c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:15.574863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8eb70-0f21-4220-a335-e7493ee90c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:15.576514 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8eb70-0f21-4220-a335-e7493ee90c39 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:15.600203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1165/2306] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:08:15.600774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2c098-e11d-44b6-b23b-ce86d5a9ca06 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:15.613109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2c098-e11d-44b6-b23b-ce86d5a9ca06 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:15.618216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e30a8-7fca-4620-bfca-ab1190e95519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:15.619117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e30a8-7fca-4620-bfca-ab1190e95519 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:15.619401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e30a8-7fca-4620-bfca-ab1190e95519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:15.619802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e30a8-7fca-4620-bfca-ab1190e95519 admin admin] PATH_INFO: `/v3/projects/e9675599cef24cfbb8d4db0a0acc13d5/users/f6a6e5882eb241d381210986e6cf94f7/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:15.637968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e30a8-7fca-4620-bfca-ab1190e95519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9675599cef24cfbb8d4db0a0acc13d5, user_id=f6a6e5882eb241d381210986e6cf94f7, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:15.640991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e30a8-7fca-4620-bfca-ab1190e95519 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:15.722754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1142/2307] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:15 2022] PUT /identity/v3/projects/e9675599cef24cfbb8d4db0a0acc13d5/users/f6a6e5882eb241d381210986e6cf94f7/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:15.735128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97298b4-2dbe-4a6d-8167-00670a5e2e3c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:15.736274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97298b4-2dbe-4a6d-8167-00670a5e2e3c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:15.736516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97298b4-2dbe-4a6d-8167-00670a5e2e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:15.758331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:15.767255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f97298b4-2dbe-4a6d-8167-00670a5e2e3c None None] MFA Rules not processed for user `f6a6e5882eb241d381210986e6cf94f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:15.768766 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f97298b4-2dbe-4a6d-8167-00670a5e2e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:15.842894 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1166/2308] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:15.854524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e30a8-7fca-4620-bfca-ab1190e95519 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:15.856195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-295e30a8-7fca-4620-bfca-ab1190e95519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:15.924273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e30a8-7fca-4620-bfca-ab1190e95519 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:15.928010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3b698-22f8-45f3-9ce7-6447e24e2542 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:15.928656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b698-22f8-45f3-9ce7-6447e24e2542 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:15.928942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b698-22f8-45f3-9ce7-6447e24e2542 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:15.929220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b698-22f8-45f3-9ce7-6447e24e2542 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:15.938068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3b698-22f8-45f3-9ce7-6447e24e2542 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:15.940099 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3b698-22f8-45f3-9ce7-6447e24e2542 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:15.949582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1143/2309] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:16.375941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bfb79-3757-4712-bc5a-e9f1a0c0d60d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:16.377032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bfb79-3757-4712-bc5a-e9f1a0c0d60d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:16.377323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bfb79-3757-4712-bc5a-e9f1a0c0d60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:16.395473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-602bfb79-3757-4712-bc5a-e9f1a0c0d60d None None] MFA Rules not processed for user `1ac8c3e3e99e412bb4a1f7b1506abce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:16.396705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-602bfb79-3757-4712-bc5a-e9f1a0c0d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:16.453320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1167/2310] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:20.413298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3b698-22f8-45f3-9ce7-6447e24e2542 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:20.415001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-32f3b698-22f8-45f3-9ce7-6447e24e2542 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:20.509427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:20.566850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3b698-22f8-45f3-9ce7-6447e24e2542 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:20.572779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:20.573661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:20.573944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:20.574348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] PATH_INFO: `/v3/users/1382feca93a34d1c90dedc23c240c4a7` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:20.575627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1382feca93a34d1c90dedc23c240c4a7)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:20.577534 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:20.832701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1382feca93a34d1c90dedc23c240c4a7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:20.840201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1382feca93a34d1c90dedc23c240c4a7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:20.853230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1382feca93a34d1c90dedc23c240c4a7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:20.892814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1382feca93a34d1c90dedc23c240c4a7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:20.893476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1382feca93a34d1c90dedc23c240c4a7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:20.903570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1144/2311] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:20 2022] DELETE /identity/v3/users/1382feca93a34d1c90dedc23c240c4a7 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:20.905350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f806fb-8213-43ec-9aa0-3e4781a3492f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:20.905808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f806fb-8213-43ec-9aa0-3e4781a3492f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:20.905956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f806fb-8213-43ec-9aa0-3e4781a3492f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:20.926772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c6f806fb-8213-43ec-9aa0-3e4781a3492f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:20.928269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c6f806fb-8213-43ec-9aa0-3e4781a3492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:21.000181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1168/2312] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:08:21.011343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:21.012766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:21.071405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc3da6-13ab-41a4-a3bf-b0d019544398 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:21.075056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ecfe9-f885-49d1-8ef3-8d8ae7b51964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:21.075703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ecfe9-f885-49d1-8ef3-8d8ae7b51964 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:21.075988 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ecfe9-f885-49d1-8ef3-8d8ae7b51964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:21.076265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ecfe9-f885-49d1-8ef3-8d8ae7b51964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:21.086917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ecfe9-f885-49d1-8ef3-8d8ae7b51964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:21.088349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ecfe9-f885-49d1-8ef3-8d8ae7b51964 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:21.098556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1145/2313] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:21.716574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f806fb-8213-43ec-9aa0-3e4781a3492f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:21.718001 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c6f806fb-8213-43ec-9aa0-3e4781a3492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:21.730390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f806fb-8213-43ec-9aa0-3e4781a3492f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:21.735020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:21.736416 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:21.736683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:21.736956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] PATH_INFO: `/v3/projects/95359cc27dc948f5a9fb45a128664b48` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:21.738171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95359cc27dc948f5a9fb45a128664b48)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:21.740102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:21.853268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:21.862091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95359cc27dc948f5a9fb45a128664b48', 'request_id': 'req-4984105a-d986-4211-9256-f3f871d506c7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:21.864990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95359cc27dc948f5a9fb45a128664b48', 'request_id': 'req-4984105a-d986-4211-9256-f3f871d506c7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:21.871154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95359cc27dc948f5a9fb45a128664b48', 'request_id': 'req-4984105a-d986-4211-9256-f3f871d506c7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:21.949750 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] The token cache is being invalidate because project 95359cc27dc948f5a9fb45a128664b48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:21.950329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:21.950908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:21.967472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1169/2314] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:21 2022] DELETE /identity/v3/projects/95359cc27dc948f5a9fb45a128664b48 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:23.308635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e5607-6b82-4ff9-b0aa-583dea067744 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:23.309084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e5607-6b82-4ff9-b0aa-583dea067744 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:23.309161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e5607-6b82-4ff9-b0aa-583dea067744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:23.328554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b42e5607-6b82-4ff9-b0aa-583dea067744 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:23.331444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b42e5607-6b82-4ff9-b0aa-583dea067744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:23.389716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1146/2315] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:08:23.402426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:23.404065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:23.497249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984105a-d986-4211-9256-f3f871d506c7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:23.504779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd32ac-1ef7-4f7c-9239-2f761e727b0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:23.506061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd32ac-1ef7-4f7c-9239-2f761e727b0c service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:23.506061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd32ac-1ef7-4f7c-9239-2f761e727b0c service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:23.506328 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd32ac-1ef7-4f7c-9239-2f761e727b0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:23.517454 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd32ac-1ef7-4f7c-9239-2f761e727b0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:23.519556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd32ac-1ef7-4f7c-9239-2f761e727b0c service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:23.535164 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1170/2316] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:08:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:08:24.127587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e5607-6b82-4ff9-b0aa-583dea067744 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:24.129161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b42e5607-6b82-4ff9-b0aa-583dea067744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:24.134788 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:24.270809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e5607-6b82-4ff9-b0aa-583dea067744 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:24.282213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:24.283257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:24.283537 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:24.283808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] PATH_INFO: `/v3/users/f8e8e759196645e3838f52014a1020c2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:24.285057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8e8e759196645e3838f52014a1020c2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:24.287392 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:24.550646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8e8e759196645e3838f52014a1020c2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:24.567165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8e8e759196645e3838f52014a1020c2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:24.601764 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8e8e759196645e3838f52014a1020c2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:24.705667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8e8e759196645e3838f52014a1020c2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:24.706347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8e8e759196645e3838f52014a1020c2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:24.727482 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1147/2317] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:24 2022] DELETE /identity/v3/users/f8e8e759196645e3838f52014a1020c2 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:24.732802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51c7ac-a63f-4a9d-bb2e-5ef1e3df9ba2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:24.733552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51c7ac-a63f-4a9d-bb2e-5ef1e3df9ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:24.734239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51c7ac-a63f-4a9d-bb2e-5ef1e3df9ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:24.841245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5f51c7ac-a63f-4a9d-bb2e-5ef1e3df9ba2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:24.841245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5f51c7ac-a63f-4a9d-bb2e-5ef1e3df9ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:24.947054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1171/2318] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:24.956996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:24.963765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:25.057685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d63263-8ced-4b46-80f7-981fcf810f0e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:25.063036 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d1d36-77f0-4db8-b3a6-1ca856430362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:25.064460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d1d36-77f0-4db8-b3a6-1ca856430362 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:25.064745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d1d36-77f0-4db8-b3a6-1ca856430362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:25.065019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d1d36-77f0-4db8-b3a6-1ca856430362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:25.078246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d1d36-77f0-4db8-b3a6-1ca856430362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:25.080516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d1d36-77f0-4db8-b3a6-1ca856430362 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:25.094225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1148/2319] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:25.780553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51c7ac-a63f-4a9d-bb2e-5ef1e3df9ba2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:25.783515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5f51c7ac-a63f-4a9d-bb2e-5ef1e3df9ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:25.807847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51c7ac-a63f-4a9d-bb2e-5ef1e3df9ba2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:25.837000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:25.843422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:25.844045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:25.844920 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] PATH_INFO: `/v3/projects/dbb9a286dde7416b92a50b729f45280f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:25.849137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb9a286dde7416b92a50b729f45280f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:25.853397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:26.032408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb9a286dde7416b92a50b729f45280f', 'request_id': 'req-ac01f269-170b-4e02-8356-3bb30e6c48ee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:26.038651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:26.041651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb9a286dde7416b92a50b729f45280f', 'request_id': 'req-ac01f269-170b-4e02-8356-3bb30e6c48ee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:26.051248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb9a286dde7416b92a50b729f45280f', 'request_id': 'req-ac01f269-170b-4e02-8356-3bb30e6c48ee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:26.133769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] The token cache is being invalidate because project dbb9a286dde7416b92a50b729f45280f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:26.134238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:26.134852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ac01f269-170b-4e02-8356-3bb30e6c48ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:26.137422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1172/2320] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:25 2022] DELETE /identity/v3/projects/dbb9a286dde7416b92a50b729f45280f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:26.142819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15436f3c-0ce8-42e3-aec6-c8bdcf3d5322 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:26.143662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15436f3c-0ce8-42e3-aec6-c8bdcf3d5322 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:26.144418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15436f3c-0ce8-42e3-aec6-c8bdcf3d5322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:26.166287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-15436f3c-0ce8-42e3-aec6-c8bdcf3d5322 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:26.167760 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-15436f3c-0ce8-42e3-aec6-c8bdcf3d5322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:26.189789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:26.235224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1149/2321] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:26.243674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5981d-8168-4212-af36-d0ba5feac3c6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:26.244310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5981d-8168-4212-af36-d0ba5feac3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:26.244746 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5981d-8168-4212-af36-d0ba5feac3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:26.264884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a3f5981d-8168-4212-af36-d0ba5feac3c6 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:26.266650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a3f5981d-8168-4212-af36-d0ba5feac3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:26.275261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1173/2322] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:26.279828 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15436f3c-0ce8-42e3-aec6-c8bdcf3d5322 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:26.291408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15436f3c-0ce8-42e3-aec6-c8bdcf3d5322 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:26.297204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff0e98-1335-47b5-ba0f-494bcea19b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:26.298696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff0e98-1335-47b5-ba0f-494bcea19b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:26.299047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff0e98-1335-47b5-ba0f-494bcea19b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:26.299327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff0e98-1335-47b5-ba0f-494bcea19b98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:26.300095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff0e98-1335-47b5-ba0f-494bcea19b98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:08:26.300400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff0e98-1335-47b5-ba0f-494bcea19b98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:26.302122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff0e98-1335-47b5-ba0f-494bcea19b98 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:26.321125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1150/2323] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:08:26.322332 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5981d-8168-4212-af36-d0ba5feac3c6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:26.332335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5981d-8168-4212-af36-d0ba5feac3c6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:26.344809 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcc66d-3f27-4b4f-a901-9cb50642c25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:26.345609 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcc66d-3f27-4b4f-a901-9cb50642c25f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:26.345962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcc66d-3f27-4b4f-a901-9cb50642c25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:26.346447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcc66d-3f27-4b4f-a901-9cb50642c25f admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:26.347206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcc66d-3f27-4b4f-a901-9cb50642c25f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:26.349197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcc66d-3f27-4b4f-a901-9cb50642c25f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:26.448825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1174/2324] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:26 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:26.452810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff0e98-1335-47b5-ba0f-494bcea19b98 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:26.464328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff0e98-1335-47b5-ba0f-494bcea19b98 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:26.527154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef0148-0255-4ff9-a224-e1b837d44244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:26.527918 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0148-0255-4ff9-a224-e1b837d44244 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:26.528114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0148-0255-4ff9-a224-e1b837d44244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:26.528420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0148-0255-4ff9-a224-e1b837d44244 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:26.529095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef0148-0255-4ff9-a224-e1b837d44244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:26.531122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef0148-0255-4ff9-a224-e1b837d44244 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:26.642595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1151/2325] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:26 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:08:26.646963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcc66d-3f27-4b4f-a901-9cb50642c25f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:26.660712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcc66d-3f27-4b4f-a901-9cb50642c25f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:26.665733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9044aeb-aae1-4416-9523-15698ddf4423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:26.666516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9044aeb-aae1-4416-9523-15698ddf4423 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:26.666801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9044aeb-aae1-4416-9523-15698ddf4423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:26.667073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9044aeb-aae1-4416-9523-15698ddf4423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:26.667808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9044aeb-aae1-4416-9523-15698ddf4423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:26.669423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9044aeb-aae1-4416-9523-15698ddf4423 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:26.691297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1175/2326] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:26.695024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef0148-0255-4ff9-a224-e1b837d44244 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:26.707195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef0148-0255-4ff9-a224-e1b837d44244 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:26.716125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b01e5-521d-4cba-b5dd-ac31039a6ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:26.717055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b01e5-521d-4cba-b5dd-ac31039a6ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:26.717515 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b01e5-521d-4cba-b5dd-ac31039a6ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:26.717937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b01e5-521d-4cba-b5dd-ac31039a6ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:26.718908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b01e5-521d-4cba-b5dd-ac31039a6ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:26.720967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b01e5-521d-4cba-b5dd-ac31039a6ac9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:26.739015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1152/2327] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:26.743690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9044aeb-aae1-4416-9523-15698ddf4423 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:26.761000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9044aeb-aae1-4416-9523-15698ddf4423 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:26.770983 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4d311-bcd0-4667-a78a-a452c6a5edef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:26.772473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4d311-bcd0-4667-a78a-a452c6a5edef admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:26.778538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4d311-bcd0-4667-a78a-a452c6a5edef admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:26.778930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4d311-bcd0-4667-a78a-a452c6a5edef admin admin] PATH_INFO: `/v3/projects/c58ebaaf4ceb417b8e916832e35faea7/users/011d6a2434464520b4affd797a395d9f/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:26.827592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4d311-bcd0-4667-a78a-a452c6a5edef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58ebaaf4ceb417b8e916832e35faea7, user_id=011d6a2434464520b4affd797a395d9f, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:26.831519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4d311-bcd0-4667-a78a-a452c6a5edef admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:26.908934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1176/2328] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:26 2022] PUT /identity/v3/projects/c58ebaaf4ceb417b8e916832e35faea7/users/011d6a2434464520b4affd797a395d9f/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:26.917884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89853a3-3db0-41b0-b68b-ce533c19a9bf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:26.918874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89853a3-3db0-41b0-b68b-ce533c19a9bf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:26.919270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89853a3-3db0-41b0-b68b-ce533c19a9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:26.979430 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f89853a3-3db0-41b0-b68b-ce533c19a9bf None None] MFA Rules not processed for user `011d6a2434464520b4affd797a395d9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:26.982208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f89853a3-3db0-41b0-b68b-ce533c19a9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:27.083225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1153/2329] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:27.093442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4d311-bcd0-4667-a78a-a452c6a5edef admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:27.095995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-72b4d311-bcd0-4667-a78a-a452c6a5edef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:27.167563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4d311-bcd0-4667-a78a-a452c6a5edef admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:27.171562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad95ed-6bb8-451e-88b0-7caa6c003aaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:27.174158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad95ed-6bb8-451e-88b0-7caa6c003aaf service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:27.174158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad95ed-6bb8-451e-88b0-7caa6c003aaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:27.174158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad95ed-6bb8-451e-88b0-7caa6c003aaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:27.196351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad95ed-6bb8-451e-88b0-7caa6c003aaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:27.197995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad95ed-6bb8-451e-88b0-7caa6c003aaf service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:27.209121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1177/2330] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:37.075400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:37.204927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=2,keystone:SELECT=58,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:38.164634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89853a3-3db0-41b0-b68b-ce533c19a9bf None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:38.166957 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f89853a3-3db0-41b0-b68b-ce533c19a9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:38.245116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89853a3-3db0-41b0-b68b-ce533c19a9bf None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:38.250495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:38.251399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:38.251886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:38.252014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] PATH_INFO: `/v3/users/f6a6e5882eb241d381210986e6cf94f7` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:38.253787 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a6e5882eb241d381210986e6cf94f7)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:38.255845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:38.443135 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a6e5882eb241d381210986e6cf94f7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:38.450280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a6e5882eb241d381210986e6cf94f7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:38.462682 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a6e5882eb241d381210986e6cf94f7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:38.531859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a6e5882eb241d381210986e6cf94f7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:38.532468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a6e5882eb241d381210986e6cf94f7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:38.539383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1154/2331] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:38 2022] DELETE /identity/v3/users/f6a6e5882eb241d381210986e6cf94f7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:38.544180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c4255-6ed1-4a35-a700-bc3720356265 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:38.544646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c4255-6ed1-4a35-a700-bc3720356265 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:38.544739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c4255-6ed1-4a35-a700-bc3720356265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:38.562342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-570c4255-6ed1-4a35-a700-bc3720356265 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:38.563561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-570c4255-6ed1-4a35-a700-bc3720356265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:38.625859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1178/2332] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:38.636418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:38.638141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:38.701111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9834e2-2d8b-41bc-bea0-3b960b7add2c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:38.705244 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587f6d7-f29b-4dea-9d91-f8857493c68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:38.706047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587f6d7-f29b-4dea-9d91-f8857493c68c service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:38.706534 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587f6d7-f29b-4dea-9d91-f8857493c68c service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:38.706819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587f6d7-f29b-4dea-9d91-f8857493c68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:38.717476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587f6d7-f29b-4dea-9d91-f8857493c68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:38.718459 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587f6d7-f29b-4dea-9d91-f8857493c68c service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:38.729912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1155/2333] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:38.811955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c4255-6ed1-4a35-a700-bc3720356265 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:38.813526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-570c4255-6ed1-4a35-a700-bc3720356265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:38.826107 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c4255-6ed1-4a35-a700-bc3720356265 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:38.830831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:38.831771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:38.832057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:38.832334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] PATH_INFO: `/v3/users/011d6a2434464520b4affd797a395d9f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:38.833534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=011d6a2434464520b4affd797a395d9f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:38.835512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:39.047761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '011d6a2434464520b4affd797a395d9f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.058506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '011d6a2434464520b4affd797a395d9f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.136375 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '011d6a2434464520b4affd797a395d9f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.137096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '011d6a2434464520b4affd797a395d9f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.138051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '011d6a2434464520b4affd797a395d9f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.158498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1179/2334] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:38 2022] DELETE /identity/v3/users/011d6a2434464520b4affd797a395d9f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:39.163923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e3276-4baf-4f1e-804a-68cc2d428978 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:39.164671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e3276-4baf-4f1e-804a-68cc2d428978 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:39.165442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e3276-4baf-4f1e-804a-68cc2d428978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:39.185321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d40e3276-4baf-4f1e-804a-68cc2d428978 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:39.186707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d40e3276-4baf-4f1e-804a-68cc2d428978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:39.210116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:39.255980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1156/2335] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:39.265648 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:39.267083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:39.319698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e3276-4baf-4f1e-804a-68cc2d428978 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:39.329020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d40e3276-4baf-4f1e-804a-68cc2d428978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:39.340105 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dcf6e-c85c-490a-93ab-0c27e2f020d0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:39.345464 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e88ece-a2df-4ad4-8dab-603b88f875f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:39.346260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e88ece-a2df-4ad4-8dab-603b88f875f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:39.347299 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e88ece-a2df-4ad4-8dab-603b88f875f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:39.347639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e88ece-a2df-4ad4-8dab-603b88f875f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:39.370615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e88ece-a2df-4ad4-8dab-603b88f875f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:39.372076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e88ece-a2df-4ad4-8dab-603b88f875f8 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:39.402753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1180/2336] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:39.404571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e88ece-a2df-4ad4-8dab-603b88f875f8 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:39.405137 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-45e88ece-a2df-4ad4-8dab-603b88f875f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:39.413565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e3276-4baf-4f1e-804a-68cc2d428978 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:39.427730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e88ece-a2df-4ad4-8dab-603b88f875f8 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:39.431096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:39.433206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:39.434340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:39.435384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] PATH_INFO: `/v3/projects/e9675599cef24cfbb8d4db0a0acc13d5` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:39.437861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9675599cef24cfbb8d4db0a0acc13d5)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:39.438690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:39.442027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:39.443307 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:39.443663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:39.444162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] PATH_INFO: `/v3/users/fd34cb0098f44ee0afa9c64857ea82da` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:39.447393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd34cb0098f44ee0afa9c64857ea82da)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:39.450588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:39.478448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:39.641092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9675599cef24cfbb8d4db0a0acc13d5', 'request_id': 'req-0f1bb1a8-b727-4a38-b309-6d7211180073'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.653814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9675599cef24cfbb8d4db0a0acc13d5', 'request_id': 'req-0f1bb1a8-b727-4a38-b309-6d7211180073'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.760779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9675599cef24cfbb8d4db0a0acc13d5', 'request_id': 'req-0f1bb1a8-b727-4a38-b309-6d7211180073'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.779532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] The token cache is being invalidate because project e9675599cef24cfbb8d4db0a0acc13d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:39.779648 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.780483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0f1bb1a8-b727-4a38-b309-6d7211180073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.782575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1157/2337] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:39 2022] DELETE /identity/v3/projects/e9675599cef24cfbb8d4db0a0acc13d5 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:39.784110 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5e703-01ce-49d5-bf91-cce5c7fdd912 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:39.784403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5e703-01ce-49d5-bf91-cce5c7fdd912 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:39.784681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5e703-01ce-49d5-bf91-cce5c7fdd912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:39.818573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-50c5e703-01ce-49d5-bf91-cce5c7fdd912 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:39.819848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-50c5e703-01ce-49d5-bf91-cce5c7fdd912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:39.834969 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd34cb0098f44ee0afa9c64857ea82da'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.871030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd34cb0098f44ee0afa9c64857ea82da'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.905236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1158/2338] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:08:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:39.921105 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5e703-01ce-49d5-bf91-cce5c7fdd912 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:39.923476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-50c5e703-01ce-49d5-bf91-cce5c7fdd912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:39.966492 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd34cb0098f44ee0afa9c64857ea82da'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.967465 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd34cb0098f44ee0afa9c64857ea82da'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:39.968347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-50b1ae46-c94b-412f-adf9-3370ea869db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd34cb0098f44ee0afa9c64857ea82da'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:40.017367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1181/2339] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:39 2022] DELETE /identity/v3/users/fd34cb0098f44ee0afa9c64857ea82da => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:40.022454 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b06318-3a76-450c-a77d-aab8cf344b03 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:40.023889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b06318-3a76-450c-a77d-aab8cf344b03 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:40.024293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b06318-3a76-450c-a77d-aab8cf344b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:40.044398 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:40.051373 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b7b06318-3a76-450c-a77d-aab8cf344b03 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:40.053093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b7b06318-3a76-450c-a77d-aab8cf344b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:40.075945 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5e703-01ce-49d5-bf91-cce5c7fdd912 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:40.160179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1182/2340] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:40.175174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b06318-3a76-450c-a77d-aab8cf344b03 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:40.177297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b7b06318-3a76-450c-a77d-aab8cf344b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:40.187328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30d42b-0e94-492d-8046-8571fd832532 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:40.188143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30d42b-0e94-492d-8046-8571fd832532 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:40.188441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30d42b-0e94-492d-8046-8571fd832532 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:40.188723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30d42b-0e94-492d-8046-8571fd832532 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:40.191262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1e30d42b-0e94-492d-8046-8571fd832532 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:40.264384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30d42b-0e94-492d-8046-8571fd832532 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:40.266575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30d42b-0e94-492d-8046-8571fd832532 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:40.279890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1159/2341] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:08:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 09:08:40.283662 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30d42b-0e94-492d-8046-8571fd832532 service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:40.285283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1e30d42b-0e94-492d-8046-8571fd832532 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:40.304447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30d42b-0e94-492d-8046-8571fd832532 service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:40.311846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:40.313941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:40.314571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:40.317457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] PATH_INFO: `/v3/projects/c58ebaaf4ceb417b8e916832e35faea7` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:40.320379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58ebaaf4ceb417b8e916832e35faea7)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:40.323717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:40.341028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b06318-3a76-450c-a77d-aab8cf344b03 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:40.345807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ea9c8-9ca5-4edd-8218-a539f6cea962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:40.347298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea9c8-9ca5-4edd-8218-a539f6cea962 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:40.347612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea9c8-9ca5-4edd-8218-a539f6cea962 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:40.348070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea9c8-9ca5-4edd-8218-a539f6cea962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:40.361273 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ea9c8-9ca5-4edd-8218-a539f6cea962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:40.363433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ea9c8-9ca5-4edd-8218-a539f6cea962 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:40.377288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1183/2342] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:40.468497 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58ebaaf4ceb417b8e916832e35faea7', 'request_id': 'req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:40.479790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58ebaaf4ceb417b8e916832e35faea7', 'request_id': 'req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:40.571620 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58ebaaf4ceb417b8e916832e35faea7', 'request_id': 'req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:40.589353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] The token cache is being invalidate because project c58ebaaf4ceb417b8e916832e35faea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:40.590186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:40.591598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ecb7dc4a-57c4-4753-8f99-cf2c9548f384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:40.593479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1160/2343] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:40 2022] DELETE /identity/v3/projects/c58ebaaf4ceb417b8e916832e35faea7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:40.600384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802dc33-d374-4754-b4be-146569bd6b7d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:40.600384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802dc33-d374-4754-b4be-146569bd6b7d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:40.600384 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802dc33-d374-4754-b4be-146569bd6b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:40.628346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c802dc33-d374-4754-b4be-146569bd6b7d None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:40.630390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c802dc33-d374-4754-b4be-146569bd6b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:40.681498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:40.790417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1184/2344] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:40.792332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c03821-664e-4291-8b67-e88896731fe1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:40.792815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c03821-664e-4291-8b67-e88896731fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:40.793303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c03821-664e-4291-8b67-e88896731fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:40.829858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-67c03821-664e-4291-8b67-e88896731fe1 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:40.832372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-67c03821-664e-4291-8b67-e88896731fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:40.854893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1161/2345] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:40.859720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802dc33-d374-4754-b4be-146569bd6b7d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:40.883949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802dc33-d374-4754-b4be-146569bd6b7d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:40.906715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab3b9b-0c19-452e-b158-50efbcba55c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:40.907512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab3b9b-0c19-452e-b158-50efbcba55c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:40.907981 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab3b9b-0c19-452e-b158-50efbcba55c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:40.908363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab3b9b-0c19-452e-b158-50efbcba55c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:40.909526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab3b9b-0c19-452e-b158-50efbcba55c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:08:40.909910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab3b9b-0c19-452e-b158-50efbcba55c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:40.911674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab3b9b-0c19-452e-b158-50efbcba55c7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:40.933529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1185/2346] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:08:40.937678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c03821-664e-4291-8b67-e88896731fe1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:40.948803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c03821-664e-4291-8b67-e88896731fe1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:40.955271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4b00d-c43c-4a36-9221-c02135ad52b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:40.956226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4b00d-c43c-4a36-9221-c02135ad52b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:40.956645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4b00d-c43c-4a36-9221-c02135ad52b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:40.956968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4b00d-c43c-4a36-9221-c02135ad52b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:40.957756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4b00d-c43c-4a36-9221-c02135ad52b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:40.960399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4b00d-c43c-4a36-9221-c02135ad52b0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:40.974310 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:41.071621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1162/2347] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:40 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:08:41.075914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab3b9b-0c19-452e-b158-50efbcba55c7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:41.087342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab3b9b-0c19-452e-b158-50efbcba55c7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:41.178607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c26cc-ef07-47ec-915b-27c31070a351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:41.180620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c26cc-ef07-47ec-915b-27c31070a351 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:41.180620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c26cc-ef07-47ec-915b-27c31070a351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:41.180620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c26cc-ef07-47ec-915b-27c31070a351 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:41.181219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c26cc-ef07-47ec-915b-27c31070a351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:41.183378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c26cc-ef07-47ec-915b-27c31070a351 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:41.279729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1186/2348] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:41 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:41.288564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4b00d-c43c-4a36-9221-c02135ad52b0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:41.307771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4b00d-c43c-4a36-9221-c02135ad52b0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:41.320290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8440b4e-69a8-4fb0-91d2-fb8e4b5b9a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:41.320903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8440b4e-69a8-4fb0-91d2-fb8e4b5b9a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:41.321194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8440b4e-69a8-4fb0-91d2-fb8e4b5b9a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:41.321985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8440b4e-69a8-4fb0-91d2-fb8e4b5b9a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:41.323477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8440b4e-69a8-4fb0-91d2-fb8e4b5b9a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:41.328249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8440b4e-69a8-4fb0-91d2-fb8e4b5b9a59 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:41.334245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c26cc-ef07-47ec-915b-27c31070a351 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:41.354729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b72c26cc-ef07-47ec-915b-27c31070a351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:41.388759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c26cc-ef07-47ec-915b-27c31070a351 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:41.405406 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:41.406924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:41.407225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:41.407511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] PATH_INFO: `/v3/projects/cf31b51aef494e8d9893321735325126` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:41.410836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf31b51aef494e8d9893321735325126)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:41.416396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:41.589629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1163/2349] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:08:41.590940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf31b51aef494e8d9893321735325126', 'request_id': 'req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:41.592715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8440b4e-69a8-4fb0-91d2-fb8e4b5b9a59 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:41.600785 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf31b51aef494e8d9893321735325126', 'request_id': 'req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:41.614711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8440b4e-69a8-4fb0-91d2-fb8e4b5b9a59 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:41.614979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf31b51aef494e8d9893321735325126', 'request_id': 'req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:41.694371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] The token cache is being invalidate because project cf31b51aef494e8d9893321735325126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:41.695084 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:41.696465 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-03e794c0-e7a4-4dc5-9e59-10d3ec931a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:41.698613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1187/2350] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:41 2022] DELETE /identity/v3/projects/cf31b51aef494e8d9893321735325126 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:41.705183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd290844-9c4d-469b-b1fa-cc4b8ef2c5f3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:41.706205 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd290844-9c4d-469b-b1fa-cc4b8ef2c5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:41.708806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd290844-9c4d-469b-b1fa-cc4b8ef2c5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:41.748721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbfadc-bf2d-4c28-9c9b-619f0b97022e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:41.749515 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbfadc-bf2d-4c28-9c9b-619f0b97022e admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:41.750059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbfadc-bf2d-4c28-9c9b-619f0b97022e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:41.750371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbfadc-bf2d-4c28-9c9b-619f0b97022e admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:41.751675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbfadc-bf2d-4c28-9c9b-619f0b97022e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:41.759466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbfadc-bf2d-4c28-9c9b-619f0b97022e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:41.775015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1164/2351] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:41.775876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbfadc-bf2d-4c28-9c9b-619f0b97022e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:41.778054 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-afbbfadc-bf2d-4c28-9c9b-619f0b97022e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:41.778271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bd290844-9c4d-469b-b1fa-cc4b8ef2c5f3 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:41.780173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bd290844-9c4d-469b-b1fa-cc4b8ef2c5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:41.796681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbfadc-bf2d-4c28-9c9b-619f0b97022e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:41.807049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1188/2352] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:41.810656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77b04a-5eaa-4b43-8e2b-7c543560db1f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:41.813326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77b04a-5eaa-4b43-8e2b-7c543560db1f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:41.813724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77b04a-5eaa-4b43-8e2b-7c543560db1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:41.816096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263faf5-59a3-4e7f-a086-5442d9403134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:41.817130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263faf5-59a3-4e7f-a086-5442d9403134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:41.817367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263faf5-59a3-4e7f-a086-5442d9403134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:41.817619 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263faf5-59a3-4e7f-a086-5442d9403134 admin admin] PATH_INFO: `/v3/projects/6d3ce5b4f6164a598a8def03e9f0f34f/users/c225d32cf2d04eb68808fe2a2465d26d/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:41.842508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263faf5-59a3-4e7f-a086-5442d9403134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3ce5b4f6164a598a8def03e9f0f34f, user_id=c225d32cf2d04eb68808fe2a2465d26d, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:41.846062 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263faf5-59a3-4e7f-a086-5442d9403134 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:41.847875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5c77b04a-5eaa-4b43-8e2b-7c543560db1f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:41.849451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5c77b04a-5eaa-4b43-8e2b-7c543560db1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:41.860689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1189/2353] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:41.865821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77b04a-5eaa-4b43-8e2b-7c543560db1f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:41.876817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77b04a-5eaa-4b43-8e2b-7c543560db1f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:41.888968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ff71e-227c-4d9a-a440-98b1ad74a4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:41.889789 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ff71e-227c-4d9a-a440-98b1ad74a4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:41.890270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ff71e-227c-4d9a-a440-98b1ad74a4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:41.890470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ff71e-227c-4d9a-a440-98b1ad74a4b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:41.891284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ff71e-227c-4d9a-a440-98b1ad74a4b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:08:41.892930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ff71e-227c-4d9a-a440-98b1ad74a4b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:41.894952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ff71e-227c-4d9a-a440-98b1ad74a4b2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:41.916250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1165/2354] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:41 2022] PUT /identity/v3/projects/6d3ce5b4f6164a598a8def03e9f0f34f/users/c225d32cf2d04eb68808fe2a2465d26d/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:41.921114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1190/2355] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:08:41.924212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263faf5-59a3-4e7f-a086-5442d9403134 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:41.927004 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1229fed-fbd9-47da-b40c-7a0a8d251d4c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:41.932000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1229fed-fbd9-47da-b40c-7a0a8d251d4c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:41.937055 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1229fed-fbd9-47da-b40c-7a0a8d251d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:41.947632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:41.980193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b1229fed-fbd9-47da-b40c-7a0a8d251d4c None None] MFA Rules not processed for user `c225d32cf2d04eb68808fe2a2465d26d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:41.981430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b1229fed-fbd9-47da-b40c-7a0a8d251d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:42.044822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263faf5-59a3-4e7f-a086-5442d9403134 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:42.050513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcd48-2197-4b09-aeb9-14dbde75ee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:42.051374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcd48-2197-4b09-aeb9-14dbde75ee0d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:42.051651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcd48-2197-4b09-aeb9-14dbde75ee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:42.051921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcd48-2197-4b09-aeb9-14dbde75ee0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:42.052638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016bcd48-2197-4b09-aeb9-14dbde75ee0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:42.054544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016bcd48-2197-4b09-aeb9-14dbde75ee0d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:42.087037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1191/2356] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:42.088264 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1229fed-fbd9-47da-b40c-7a0a8d251d4c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:42.107190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1229fed-fbd9-47da-b40c-7a0a8d251d4c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:42.123944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036d862-1a9a-4666-8dc2-9119835f7ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:42.129359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036d862-1a9a-4666-8dc2-9119835f7ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:42.129359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036d862-1a9a-4666-8dc2-9119835f7ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:42.129359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036d862-1a9a-4666-8dc2-9119835f7ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:42.129359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036d862-1a9a-4666-8dc2-9119835f7ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:42.129732 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036d862-1a9a-4666-8dc2-9119835f7ff8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:42.157928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1166/2357] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:41 2022] POST /identity/v3/projects => generated 379 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:42.162669 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcd48-2197-4b09-aeb9-14dbde75ee0d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:42.214408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:42.246197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1192/2358] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:42 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:42.247237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036d862-1a9a-4666-8dc2-9119835f7ff8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:42.265708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcd48-2197-4b09-aeb9-14dbde75ee0d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:42.368447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b98c4-d2ff-4bfc-b30c-8fbd48f95904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:42.369269 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b98c4-d2ff-4bfc-b30c-8fbd48f95904 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:42.369511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b98c4-d2ff-4bfc-b30c-8fbd48f95904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:42.369908 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b98c4-d2ff-4bfc-b30c-8fbd48f95904 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:42.373267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b98c4-d2ff-4bfc-b30c-8fbd48f95904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:42.373451 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b98c4-d2ff-4bfc-b30c-8fbd48f95904 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:42.458421 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036d862-1a9a-4666-8dc2-9119835f7ff8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:42.469296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e9961-930a-4799-a1e3-67a856e19b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:42.469296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e9961-930a-4799-a1e3-67a856e19b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:42.469296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e9961-930a-4799-a1e3-67a856e19b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:42.469296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e9961-930a-4799-a1e3-67a856e19b81 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:42.469676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e9961-930a-4799-a1e3-67a856e19b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:42.469676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e9961-930a-4799-a1e3-67a856e19b81 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:42.890430 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1167/2359] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:42 2022] POST /identity/v3/users => generated 290 bytes in 728 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:42.891655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b98c4-d2ff-4bfc-b30c-8fbd48f95904 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:42.911519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1193/2360] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:42 2022] POST /identity/v3/users => generated 293 bytes in 665 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:08:42.917718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e9961-930a-4799-a1e3-67a856e19b81 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:42.920526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b98c4-d2ff-4bfc-b30c-8fbd48f95904 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:42.932137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a6feb-8788-482e-ba5c-7f284610ce33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:42.932973 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e9961-930a-4799-a1e3-67a856e19b81 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:42.933298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a6feb-8788-482e-ba5c-7f284610ce33 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:42.933832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a6feb-8788-482e-ba5c-7f284610ce33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:42.934297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a6feb-8788-482e-ba5c-7f284610ce33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:42.935295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a6feb-8788-482e-ba5c-7f284610ce33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:42.937261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a6feb-8788-482e-ba5c-7f284610ce33 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:42.939387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e51744-bfdf-4c72-a440-d94c1b23c19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:42.941056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e51744-bfdf-4c72-a440-d94c1b23c19c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:42.942906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e51744-bfdf-4c72-a440-d94c1b23c19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:42.944723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e51744-bfdf-4c72-a440-d94c1b23c19c admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:42.944723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e51744-bfdf-4c72-a440-d94c1b23c19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:42.946221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e51744-bfdf-4c72-a440-d94c1b23c19c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:42.954961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1168/2361] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:42.959982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a6feb-8788-482e-ba5c-7f284610ce33 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:42.964880 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1194/2362] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:08:42.966242 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e51744-bfdf-4c72-a440-d94c1b23c19c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:42.972232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a6feb-8788-482e-ba5c-7f284610ce33 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:42.979498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0adfd-8607-4e3b-a1b5-2f998a27e462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:42.980414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0adfd-8607-4e3b-a1b5-2f998a27e462 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:42.980850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e51744-bfdf-4c72-a440-d94c1b23c19c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:42.981030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0adfd-8607-4e3b-a1b5-2f998a27e462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:42.981565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0adfd-8607-4e3b-a1b5-2f998a27e462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:42.982618 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0adfd-8607-4e3b-a1b5-2f998a27e462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:42.984810 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0adfd-8607-4e3b-a1b5-2f998a27e462 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:42.986399 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a0ac6-804d-408b-b14b-20b669eba80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:42.988272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0ac6-804d-408b-b14b-20b669eba80d admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:42.989300 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0ac6-804d-408b-b14b-20b669eba80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:42.989591 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a0ac6-804d-408b-b14b-20b669eba80d admin admin] PATH_INFO: `/v3/projects/dc6cc5cb8f2c4cb1931b6254edd9b489/users/1b4d70b5d435458a8e2aaacd95c95602/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:43.000985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1169/2363] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:43.005935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0adfd-8607-4e3b-a1b5-2f998a27e462 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:43.010423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a0ac6-804d-408b-b14b-20b669eba80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc6cc5cb8f2c4cb1931b6254edd9b489, user_id=1b4d70b5d435458a8e2aaacd95c95602, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:43.014688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a0ac6-804d-408b-b14b-20b669eba80d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:43.017985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0adfd-8607-4e3b-a1b5-2f998a27e462 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:43.024001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a3e78-e273-4529-9e87-5f03a94ad63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:43.025184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a3e78-e273-4529-9e87-5f03a94ad63c admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:43.025621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a3e78-e273-4529-9e87-5f03a94ad63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:43.026211 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a3e78-e273-4529-9e87-5f03a94ad63c admin admin] PATH_INFO: `/v3/projects/02051a1f629a48199bb1eb504c747294/users/6f101dab08684c2483f67a9357efeae8/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:43.066089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a3e78-e273-4529-9e87-5f03a94ad63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=02051a1f629a48199bb1eb504c747294, user_id=6f101dab08684c2483f67a9357efeae8, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:43.068740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a3e78-e273-4529-9e87-5f03a94ad63c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:43.117206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1195/2364] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:42 2022] PUT /identity/v3/projects/dc6cc5cb8f2c4cb1931b6254edd9b489/users/1b4d70b5d435458a8e2aaacd95c95602/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:43.122460 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a0ac6-804d-408b-b14b-20b669eba80d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:43.163263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1170/2365] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:43 2022] PUT /identity/v3/projects/02051a1f629a48199bb1eb504c747294/users/6f101dab08684c2483f67a9357efeae8/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:43.181509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f414c-cacc-4582-8076-8d4c5b27ed49 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:43.183674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f414c-cacc-4582-8076-8d4c5b27ed49 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:43.184287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f414c-cacc-4582-8076-8d4c5b27ed49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:43.210846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:43.256833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4c0f414c-cacc-4582-8076-8d4c5b27ed49 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:43.256833 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4c0f414c-cacc-4582-8076-8d4c5b27ed49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:43.283612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a0ac6-804d-408b-b14b-20b669eba80d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:43.352418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1171/2366] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:43.353334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f414c-cacc-4582-8076-8d4c5b27ed49 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:43.357374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4c0f414c-cacc-4582-8076-8d4c5b27ed49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:43.397578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:43.405248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c4339-33f3-450f-87a1-cb21adcf922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:43.405985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c4339-33f3-450f-87a1-cb21adcf922f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:43.406365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c4339-33f3-450f-87a1-cb21adcf922f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:43.406720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c4339-33f3-450f-87a1-cb21adcf922f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:43.407526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c4339-33f3-450f-87a1-cb21adcf922f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:43.410002 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c4339-33f3-450f-87a1-cb21adcf922f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:43.423664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1196/2367] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:43.428283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c4339-33f3-450f-87a1-cb21adcf922f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:43.444349 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c4339-33f3-450f-87a1-cb21adcf922f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:43.450256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd081fa-2d14-423e-8442-1da0e358da35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:43.451889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd081fa-2d14-423e-8442-1da0e358da35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:43.451889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd081fa-2d14-423e-8442-1da0e358da35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:43.452124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd081fa-2d14-423e-8442-1da0e358da35 admin admin] PATH_INFO: `/v3/projects/dc6cc5cb8f2c4cb1931b6254edd9b489/users/1b4d70b5d435458a8e2aaacd95c95602/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:43.455424 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd081fa-2d14-423e-8442-1da0e358da35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc6cc5cb8f2c4cb1931b6254edd9b489, user_id=1b4d70b5d435458a8e2aaacd95c95602, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:43.458829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd081fa-2d14-423e-8442-1da0e358da35 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:43.473766 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1197/2368] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:43 2022] PUT /identity/v3/projects/dc6cc5cb8f2c4cb1931b6254edd9b489/users/1b4d70b5d435458a8e2aaacd95c95602/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:43.475243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f414c-cacc-4582-8076-8d4c5b27ed49 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:43.491931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099256ce-1144-4613-a2ef-b4517454029e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:43.492516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099256ce-1144-4613-a2ef-b4517454029e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:43.492951 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099256ce-1144-4613-a2ef-b4517454029e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:43.557561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-099256ce-1144-4613-a2ef-b4517454029e None None] MFA Rules not processed for user `1b4d70b5d435458a8e2aaacd95c95602`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:43.558814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-099256ce-1144-4613-a2ef-b4517454029e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:43.573344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349cd31-3a33-46f3-90da-240f50d15138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:43.574302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349cd31-3a33-46f3-90da-240f50d15138 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:43.574767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349cd31-3a33-46f3-90da-240f50d15138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:43.575194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349cd31-3a33-46f3-90da-240f50d15138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:43.655489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349cd31-3a33-46f3-90da-240f50d15138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:43.657519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349cd31-3a33-46f3-90da-240f50d15138 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:43.671148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1172/2369] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:43.697712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1198/2370] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:43.711703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349cd31-3a33-46f3-90da-240f50d15138 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:43.713370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3349cd31-3a33-46f3-90da-240f50d15138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:43.824292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349cd31-3a33-46f3-90da-240f50d15138 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:43.828368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6c2e4-8fca-498a-becb-6c71b977ba85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:43.829492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6c2e4-8fca-498a-becb-6c71b977ba85 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:43.829774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6c2e4-8fca-498a-becb-6c71b977ba85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:43.830464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6c2e4-8fca-498a-becb-6c71b977ba85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:43.917737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6c2e4-8fca-498a-becb-6c71b977ba85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:43.919545 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6c2e4-8fca-498a-becb-6c71b977ba85 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:43.936438 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1173/2371] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:44.069487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:47.509616 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099256ce-1144-4613-a2ef-b4517454029e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:47.534414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099256ce-1144-4613-a2ef-b4517454029e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:47.545393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:47.547087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:47.547558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:47.548075 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] PATH_INFO: `/v3/users/c225d32cf2d04eb68808fe2a2465d26d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:47.551376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c225d32cf2d04eb68808fe2a2465d26d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:47.553650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:47.757327 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:47.773133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c225d32cf2d04eb68808fe2a2465d26d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:47.783786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c225d32cf2d04eb68808fe2a2465d26d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:47.863314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c225d32cf2d04eb68808fe2a2465d26d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:47.864633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c225d32cf2d04eb68808fe2a2465d26d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:47.865191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c225d32cf2d04eb68808fe2a2465d26d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:47.901560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1199/2372] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:47 2022] DELETE /identity/v3/users/c225d32cf2d04eb68808fe2a2465d26d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:47.906583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff3314-16f4-4dab-898f-cecc402330ee None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:47.907164 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff3314-16f4-4dab-898f-cecc402330ee None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:47.907164 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff3314-16f4-4dab-898f-cecc402330ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:47.935948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-17ff3314-16f4-4dab-898f-cecc402330ee None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:47.937431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-17ff3314-16f4-4dab-898f-cecc402330ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:48.043920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1174/2373] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:48.053794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:48.053794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:48.175143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a5013-bd14-411b-864c-402ef4a44567 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:48.180521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27c1cb-e3f0-46bc-9ad4-92af362a2d72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:48.180521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27c1cb-e3f0-46bc-9ad4-92af362a2d72 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:48.181175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27c1cb-e3f0-46bc-9ad4-92af362a2d72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:48.181410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27c1cb-e3f0-46bc-9ad4-92af362a2d72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:48.210246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27c1cb-e3f0-46bc-9ad4-92af362a2d72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:48.212940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27c1cb-e3f0-46bc-9ad4-92af362a2d72 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:48.233133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1200/2374] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:08:49.854227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff3314-16f4-4dab-898f-cecc402330ee None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:49.856673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-17ff3314-16f4-4dab-898f-cecc402330ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:49.876091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff3314-16f4-4dab-898f-cecc402330ee None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:49.887823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:49.889123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:49.889571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:49.889880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] PATH_INFO: `/v3/projects/6d3ce5b4f6164a598a8def03e9f0f34f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:49.891308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3ce5b4f6164a598a8def03e9f0f34f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:49.894989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:50.019956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3ce5b4f6164a598a8def03e9f0f34f', 'request_id': 'req-70aa0e09-a742-460e-9823-72d2d44539c4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:50.034787 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3ce5b4f6164a598a8def03e9f0f34f', 'request_id': 'req-70aa0e09-a742-460e-9823-72d2d44539c4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:50.146968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3ce5b4f6164a598a8def03e9f0f34f', 'request_id': 'req-70aa0e09-a742-460e-9823-72d2d44539c4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:50.176047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] The token cache is being invalidate because project 6d3ce5b4f6164a598a8def03e9f0f34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:50.176551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:50.177374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70aa0e09-a742-460e-9823-72d2d44539c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:50.179399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1175/2375] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:49 2022] DELETE /identity/v3/projects/6d3ce5b4f6164a598a8def03e9f0f34f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:50.183203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27c1cb-e3f0-46bc-9ad4-92af362a2d72 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:50.198474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27c1cb-e3f0-46bc-9ad4-92af362a2d72 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:50.329018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:50.329526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:50.329818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:50.330110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] PATH_INFO: `/v3/users/1b4d70b5d435458a8e2aaacd95c95602` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:50.335563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4d70b5d435458a8e2aaacd95c95602)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:50.346326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:50.686431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab44d3a-d308-4195-877d-71374fc28485 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:50.686718 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab44d3a-d308-4195-877d-71374fc28485 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:50.686991 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab44d3a-d308-4195-877d-71374fc28485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:50.702724 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4d70b5d435458a8e2aaacd95c95602'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:50.719838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4d70b5d435458a8e2aaacd95c95602'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:50.724247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7ab44d3a-d308-4195-877d-71374fc28485 None None] MFA Rules not processed for user `d9bffb24cf26486a9a8947882b52ec98`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:50.725335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:50.726250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7ab44d3a-d308-4195-877d-71374fc28485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:50.811541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4d70b5d435458a8e2aaacd95c95602'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:50.812679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4d70b5d435458a8e2aaacd95c95602'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:50.813572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d687dfd5-a86b-465a-914f-51028567d508 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4d70b5d435458a8e2aaacd95c95602'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:50.849523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1176/2376] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:50.865304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1201/2377] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:08:50 2022] DELETE /identity/v3/users/1b4d70b5d435458a8e2aaacd95c95602 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:51.619423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab44d3a-d308-4195-877d-71374fc28485 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:51.620911 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7ab44d3a-d308-4195-877d-71374fc28485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:51.651572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab44d3a-d308-4195-877d-71374fc28485 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:51.682051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:51.750368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:51.751341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:51.751625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:51.751891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] PATH_INFO: `/v3/projects/dc6cc5cb8f2c4cb1931b6254edd9b489` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:51.753868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc6cc5cb8f2c4cb1931b6254edd9b489)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:51.755822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:51.966255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc6cc5cb8f2c4cb1931b6254edd9b489', 'request_id': 'req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:51.979197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc6cc5cb8f2c4cb1931b6254edd9b489', 'request_id': 'req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:52.050084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc6cc5cb8f2c4cb1931b6254edd9b489', 'request_id': 'req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:52.076581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] The token cache is being invalidate because project dc6cc5cb8f2c4cb1931b6254edd9b489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:08:52.079110 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:52.080780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-52a4b0f7-c6d7-45f8-a977-c21cb990b0f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:08:52.086530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1177/2378] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:08:51 2022] DELETE /identity/v3/projects/dc6cc5cb8f2c4cb1931b6254edd9b489 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:52.091123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264b16c-59fb-4a46-aec0-4f14d8197563 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:52.091611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264b16c-59fb-4a46-aec0-4f14d8197563 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:52.091992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264b16c-59fb-4a46-aec0-4f14d8197563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:52.163655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b264b16c-59fb-4a46-aec0-4f14d8197563 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:52.168020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b264b16c-59fb-4a46-aec0-4f14d8197563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:52.309305 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1202/2379] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:52.317521 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101053bc-f541-447a-a73e-e4fe640f1b2c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:52.317799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101053bc-f541-447a-a73e-e4fe640f1b2c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:52.318267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101053bc-f541-447a-a73e-e4fe640f1b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:52.342485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-101053bc-f541-447a-a73e-e4fe640f1b2c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:52.344235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-101053bc-f541-447a-a73e-e4fe640f1b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:52.363350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1178/2380] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:52.364304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264b16c-59fb-4a46-aec0-4f14d8197563 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:52.383318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264b16c-59fb-4a46-aec0-4f14d8197563 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:52.389945 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbc884-05c9-4363-9fda-fbf9b728f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:52.393260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbc884-05c9-4363-9fda-fbf9b728f2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:52.393260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbc884-05c9-4363-9fda-fbf9b728f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:52.393260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbc884-05c9-4363-9fda-fbf9b728f2b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:52.393260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbc884-05c9-4363-9fda-fbf9b728f2b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:08:52.393260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbc884-05c9-4363-9fda-fbf9b728f2b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:52.394198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbc884-05c9-4363-9fda-fbf9b728f2b9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:52.417892 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1203/2381] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:08:52.419866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101053bc-f541-447a-a73e-e4fe640f1b2c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:52.431906 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101053bc-f541-447a-a73e-e4fe640f1b2c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:52.446879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1ca4c-67e3-4b66-a79a-1ff8a7af46dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:52.447665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1ca4c-67e3-4b66-a79a-1ff8a7af46dc admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:52.448171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1ca4c-67e3-4b66-a79a-1ff8a7af46dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:52.448561 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1ca4c-67e3-4b66-a79a-1ff8a7af46dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:52.449347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b1ca4c-67e3-4b66-a79a-1ff8a7af46dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:52.452348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b1ca4c-67e3-4b66-a79a-1ff8a7af46dc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:52.618967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1179/2382] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:52 2022] POST /identity/v3/projects => generated 369 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:52.622398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbc884-05c9-4363-9fda-fbf9b728f2b9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:52.638121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbc884-05c9-4363-9fda-fbf9b728f2b9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:52.778523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462f6c8-4469-4448-bd7c-a73807ebc6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:52.779415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462f6c8-4469-4448-bd7c-a73807ebc6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:52.780013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462f6c8-4469-4448-bd7c-a73807ebc6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:52.780546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462f6c8-4469-4448-bd7c-a73807ebc6d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:52.782069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462f6c8-4469-4448-bd7c-a73807ebc6d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:52.785818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462f6c8-4469-4448-bd7c-a73807ebc6d0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:52.919852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1204/2383] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:52 2022] POST /identity/v3/users => generated 285 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:52.923500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1ca4c-67e3-4b66-a79a-1ff8a7af46dc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:52.963470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1ca4c-67e3-4b66-a79a-1ff8a7af46dc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:52.972072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48b19a-6468-40be-852c-e71368134393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:52.973176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48b19a-6468-40be-852c-e71368134393 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:52.973630 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48b19a-6468-40be-852c-e71368134393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:52.974200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48b19a-6468-40be-852c-e71368134393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:52.975396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48b19a-6468-40be-852c-e71368134393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:52.983281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48b19a-6468-40be-852c-e71368134393 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:53.004731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1180/2384] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:53.008174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462f6c8-4469-4448-bd7c-a73807ebc6d0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:53.028685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462f6c8-4469-4448-bd7c-a73807ebc6d0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:53.050446 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cdfc0-fa8a-4adc-8ca8-d2bea7a986d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:53.060007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cdfc0-fa8a-4adc-8ca8-d2bea7a986d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:53.060007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cdfc0-fa8a-4adc-8ca8-d2bea7a986d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:53.060007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cdfc0-fa8a-4adc-8ca8-d2bea7a986d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:53.060007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cdfc0-fa8a-4adc-8ca8-d2bea7a986d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:53.060007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cdfc0-fa8a-4adc-8ca8-d2bea7a986d2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:53.088611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1205/2385] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:53.091644 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48b19a-6468-40be-852c-e71368134393 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:53.107258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48b19a-6468-40be-852c-e71368134393 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:53.114105 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7d4f-b865-4875-a5b9-3d70a4855179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:53.115829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7d4f-b865-4875-a5b9-3d70a4855179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:53.116192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7d4f-b865-4875-a5b9-3d70a4855179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:53.116631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d7d4f-b865-4875-a5b9-3d70a4855179 admin admin] PATH_INFO: `/v3/projects/0df96ccaf9c848519ceda061b574228f/users/144831778810401d9c11b579b1ccad54/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:53.154538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7d4f-b865-4875-a5b9-3d70a4855179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df96ccaf9c848519ceda061b574228f, user_id=144831778810401d9c11b579b1ccad54, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:53.158909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d7d4f-b865-4875-a5b9-3d70a4855179 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:53.304485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1181/2386] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:53 2022] PUT /identity/v3/projects/0df96ccaf9c848519ceda061b574228f/users/144831778810401d9c11b579b1ccad54/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:53.316449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a5d38-8af8-47e8-87e9-0d2761751e55 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:53.318022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a5d38-8af8-47e8-87e9-0d2761751e55 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:53.318893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a5d38-8af8-47e8-87e9-0d2761751e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:53.386637 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-588a5d38-8af8-47e8-87e9-0d2761751e55 None None] MFA Rules not processed for user `144831778810401d9c11b579b1ccad54`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:53.388318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-588a5d38-8af8-47e8-87e9-0d2761751e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:53.457549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:53.541212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1206/2387] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:53.542256 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7d4f-b865-4875-a5b9-3d70a4855179 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:53.544207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bc6d7d4f-b865-4875-a5b9-3d70a4855179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:53.648847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:08:53.679471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d7d4f-b865-4875-a5b9-3d70a4855179 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:53.687278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f64df-75e2-4f11-a6fb-88cd55267a0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf67fa55f4d64b7ebeeb1490453661e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:53.688064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f64df-75e2-4f11-a6fb-88cd55267a0a service glance] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:53.688395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f64df-75e2-4f11-a6fb-88cd55267a0a service glance] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:53.688727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f64df-75e2-4f11-a6fb-88cd55267a0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:53.705834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f64df-75e2-4f11-a6fb-88cd55267a0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:53.708777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f64df-75e2-4f11-a6fb-88cd55267a0a service glance] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:53.733471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1182/2388] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:08:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:08:54.256832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb8616-e1ea-4957-a5a4-dcc76f1caac4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:54.257363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb8616-e1ea-4957-a5a4-dcc76f1caac4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:54.257761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb8616-e1ea-4957-a5a4-dcc76f1caac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:54.301926 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a0fb8616-e1ea-4957-a5a4-dcc76f1caac4 None None] MFA Rules not processed for user `6f101dab08684c2483f67a9357efeae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:08:54.303854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a0fb8616-e1ea-4957-a5a4-dcc76f1caac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:54.423636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1207/2389] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:08:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:08:54.428048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f64df-75e2-4f11-a6fb-88cd55267a0a service glance] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:54.430869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c46f64df-75e2-4f11-a6fb-88cd55267a0a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:08:54.597841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f64df-75e2-4f11-a6fb-88cd55267a0a service glance] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:54.602789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0e3db-0d5d-4ce7-a8cd-b0a3bd8a69dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:54.604055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0e3db-0d5d-4ce7-a8cd-b0a3bd8a69dc admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:54.604586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0e3db-0d5d-4ce7-a8cd-b0a3bd8a69dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:54.604868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0e3db-0d5d-4ce7-a8cd-b0a3bd8a69dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:54.606692 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0e3db-0d5d-4ce7-a8cd-b0a3bd8a69dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:54.608727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0e3db-0d5d-4ce7-a8cd-b0a3bd8a69dc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:54.744816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1183/2390] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:08:54 2022] POST /identity/v3/projects => generated 377 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:08:54.747117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb8616-e1ea-4957-a5a4-dcc76f1caac4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:54.777176 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb8616-e1ea-4957-a5a4-dcc76f1caac4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:54.910074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3700d-c31c-4eb3-9fef-575514c94929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:54.913653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3700d-c31c-4eb3-9fef-575514c94929 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:54.913653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3700d-c31c-4eb3-9fef-575514c94929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:54.913653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3700d-c31c-4eb3-9fef-575514c94929 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:54.913653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3700d-c31c-4eb3-9fef-575514c94929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:54.915884 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3700d-c31c-4eb3-9fef-575514c94929 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:55.087258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1208/2391] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:08:54 2022] POST /identity/v3/users => generated 295 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:08:55.099728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0e3db-0d5d-4ce7-a8cd-b0a3bd8a69dc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:55.107484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0e3db-0d5d-4ce7-a8cd-b0a3bd8a69dc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:55.112834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ac262-be47-4198-acc3-763bccc78e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:55.117550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ac262-be47-4198-acc3-763bccc78e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:55.117550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ac262-be47-4198-acc3-763bccc78e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:55.117550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ac262-be47-4198-acc3-763bccc78e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:55.117550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ac262-be47-4198-acc3-763bccc78e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:55.117550 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ac262-be47-4198-acc3-763bccc78e2d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:55.129289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1184/2392] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:55.135207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3700d-c31c-4eb3-9fef-575514c94929 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:55.153506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3700d-c31c-4eb3-9fef-575514c94929 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:55.161410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1da307-6683-4ce0-aaec-f42041434f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:55.162909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1da307-6683-4ce0-aaec-f42041434f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:55.163522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1da307-6683-4ce0-aaec-f42041434f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:55.164094 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1da307-6683-4ce0-aaec-f42041434f31 admin admin] PATH_INFO: `/v3/projects/b16f471ef00f4ee5a3afe56bfdba8ae9/users/03e63690d4ee4947b647239d2ba98d2d/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:55.185280 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1da307-6683-4ce0-aaec-f42041434f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16f471ef00f4ee5a3afe56bfdba8ae9, user_id=03e63690d4ee4947b647239d2ba98d2d, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:55.188170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1da307-6683-4ce0-aaec-f42041434f31 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:55.852428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1209/2393] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:55 2022] PUT /identity/v3/projects/b16f471ef00f4ee5a3afe56bfdba8ae9/users/03e63690d4ee4947b647239d2ba98d2d/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:08:55.856486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ac262-be47-4198-acc3-763bccc78e2d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:55.867421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ac262-be47-4198-acc3-763bccc78e2d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:55.985934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85d074-fb89-469d-aab5-5c8d35ba274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:55.987714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85d074-fb89-469d-aab5-5c8d35ba274e admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:55.988127 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85d074-fb89-469d-aab5-5c8d35ba274e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:55.988402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85d074-fb89-469d-aab5-5c8d35ba274e admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:55.989140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85d074-fb89-469d-aab5-5c8d35ba274e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:55.991824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85d074-fb89-469d-aab5-5c8d35ba274e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:56.035804 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1185/2394] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:08:56.036773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1da307-6683-4ce0-aaec-f42041434f31 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:08:56.061769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1da307-6683-4ce0-aaec-f42041434f31 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:08:56.070752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77afd7ac-51ed-475c-a837-82a03db2006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:08:56.073545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afd7ac-51ed-475c-a837-82a03db2006b admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:08:56.074092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afd7ac-51ed-475c-a837-82a03db2006b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:08:56.075609 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afd7ac-51ed-475c-a837-82a03db2006b admin admin] PATH_INFO: `/v3/projects/b16f471ef00f4ee5a3afe56bfdba8ae9/users/03e63690d4ee4947b647239d2ba98d2d/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:08:56.082507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77afd7ac-51ed-475c-a837-82a03db2006b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16f471ef00f4ee5a3afe56bfdba8ae9, user_id=03e63690d4ee4947b647239d2ba98d2d, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:08:56.086279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77afd7ac-51ed-475c-a837-82a03db2006b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:08:56.122749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1210/2395] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:08:56 2022] PUT /identity/v3/projects/b16f471ef00f4ee5a3afe56bfdba8ae9/users/03e63690d4ee4947b647239d2ba98d2d/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:09:01.209533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85d074-fb89-469d-aab5-5c8d35ba274e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:01.211753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4c85d074-fb89-469d-aab5-5c8d35ba274e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:01.291562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85d074-fb89-469d-aab5-5c8d35ba274e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:01.307040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:01.376213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee714fe-07f6-4877-b22e-f2d1e57a0b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:01.377051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee714fe-07f6-4877-b22e-f2d1e57a0b9d service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:01.377508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee714fe-07f6-4877-b22e-f2d1e57a0b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:01.377959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee714fe-07f6-4877-b22e-f2d1e57a0b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:01.380286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5ee714fe-07f6-4877-b22e-f2d1e57a0b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:01.456818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee714fe-07f6-4877-b22e-f2d1e57a0b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:01.458903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee714fe-07f6-4877-b22e-f2d1e57a0b9d service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:01.469771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1186/2396] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:09:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:09:05.623498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17731613-5690-4d01-b9d8-2cadf07ad821 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:05.624031 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17731613-5690-4d01-b9d8-2cadf07ad821 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:05.624118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17731613-5690-4d01-b9d8-2cadf07ad821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:05.669529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:05.674220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-17731613-5690-4d01-b9d8-2cadf07ad821 None None] MFA Rules not processed for user `03e63690d4ee4947b647239d2ba98d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:05.676161 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-17731613-5690-4d01-b9d8-2cadf07ad821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:05.807052 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1211/2397] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:09:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:09:05.833437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee714fe-07f6-4877-b22e-f2d1e57a0b9d service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:05.835070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5ee714fe-07f6-4877-b22e-f2d1e57a0b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:05.960879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee714fe-07f6-4877-b22e-f2d1e57a0b9d service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:05.967273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e67df-da17-4c40-938d-f01c19b6c8e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:05.967986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e67df-da17-4c40-938d-f01c19b6c8e6 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:05.968263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e67df-da17-4c40-938d-f01c19b6c8e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:05.968531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e67df-da17-4c40-938d-f01c19b6c8e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:06.054633 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e67df-da17-4c40-938d-f01c19b6c8e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:06.056488 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e67df-da17-4c40-938d-f01c19b6c8e6 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:06.060231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:06.066578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1187/2398] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:09:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:09:07.051295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71a76a-4e36-47bd-93db-ac9236badd85 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:07.051806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71a76a-4e36-47bd-93db-ac9236badd85 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:07.051910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71a76a-4e36-47bd-93db-ac9236badd85 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:07.055021 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1212/2399] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 09:09:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:09:07.061300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c2ac-204d-4360-9de9-082b09165bb4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:07.062091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c2ac-204d-4360-9de9-082b09165bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:07.062445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c2ac-204d-4360-9de9-082b09165bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:07.095628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-96b3c2ac-204d-4360-9de9-082b09165bb4 None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:07.097515 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-96b3c2ac-204d-4360-9de9-082b09165bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:07.110227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1188/2400] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 09:09:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:09:07.122097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71a76a-4e36-47bd-93db-ac9236badd85 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:07.124241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2d71a76a-4e36-47bd-93db-ac9236badd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:07.192352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71a76a-4e36-47bd-93db-ac9236badd85 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:07.195997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae5d4f-a5e4-4896-b09f-10896debcad3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf67fa55f4d64b7ebeeb1490453661e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:07.196677 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae5d4f-a5e4-4896-b09f-10896debcad3 service glance] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:07.196959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae5d4f-a5e4-4896-b09f-10896debcad3 service glance] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:07.197237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae5d4f-a5e4-4896-b09f-10896debcad3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:07.208936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae5d4f-a5e4-4896-b09f-10896debcad3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:07.210420 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae5d4f-a5e4-4896-b09f-10896debcad3 service glance] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:07.223091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1213/2401] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:09:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:09:07.470671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236704f9-c55f-4a4e-a546-068cba43b536 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:07.470957 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236704f9-c55f-4a4e-a546-068cba43b536 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:07.471223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236704f9-c55f-4a4e-a546-068cba43b536 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:07.472567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1189/2402] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 09:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:09:07.477511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f206d7-0f90-4048-b9a2-1b7c90ccb1b0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:07.477889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f206d7-0f90-4048-b9a2-1b7c90ccb1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:07.478208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f206d7-0f90-4048-b9a2-1b7c90ccb1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:07.500943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-10f206d7-0f90-4048-b9a2-1b7c90ccb1b0 None None] MFA Rules not processed for user `12824152139841138f457dfe5c9758e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:07.504791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-10f206d7-0f90-4048-b9a2-1b7c90ccb1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:07.552438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1214/2403] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 09:09:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:09:07.552747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236704f9-c55f-4a4e-a546-068cba43b536 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:07.553915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-236704f9-c55f-4a4e-a546-068cba43b536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:07.570983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236704f9-c55f-4a4e-a546-068cba43b536 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:07.575337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f70fb-67e2-4e8c-a72c-02bae8031a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:07.576487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f70fb-67e2-4e8c-a72c-02bae8031a8f service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:07.576963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f70fb-67e2-4e8c-a72c-02bae8031a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:07.577418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f70fb-67e2-4e8c-a72c-02bae8031a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:07.589494 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f70fb-67e2-4e8c-a72c-02bae8031a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:07.591558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f70fb-67e2-4e8c-a72c-02bae8031a8f service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:07.604020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1190/2404] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:09:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:09:07.738440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804beaa-6317-437c-a69b-056218bf7ac7 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:07.738807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804beaa-6317-437c-a69b-056218bf7ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:07.739154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804beaa-6317-437c-a69b-056218bf7ac7 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:07.740789 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1215/2405] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 09:09:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:09:07.753167 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b7ba7-82a9-46d6-8ff9-44e37456f4c7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:07.753565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b7ba7-82a9-46d6-8ff9-44e37456f4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:07.753915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b7ba7-82a9-46d6-8ff9-44e37456f4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:07.800068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a38b7ba7-82a9-46d6-8ff9-44e37456f4c7 None None] MFA Rules not processed for user `8af26e036f4e466ebb923771c1f5f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:07.802568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a38b7ba7-82a9-46d6-8ff9-44e37456f4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:07.833928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1191/2406] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 09:09:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:09:07.876383 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804beaa-6317-437c-a69b-056218bf7ac7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:07.903369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804beaa-6317-437c-a69b-056218bf7ac7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:07.922889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba084f-b634-43ff-a736-683dcd30e362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:07.923772 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba084f-b634-43ff-a736-683dcd30e362 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:07.924054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba084f-b634-43ff-a736-683dcd30e362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:07.924330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba084f-b634-43ff-a736-683dcd30e362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:07.936122 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba084f-b634-43ff-a736-683dcd30e362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:07.937566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba084f-b634-43ff-a736-683dcd30e362 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:07.951329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1216/2407] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:09:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:09:11.257745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b7ba7-82a9-46d6-8ff9-44e37456f4c7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:11.259771 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a38b7ba7-82a9-46d6-8ff9-44e37456f4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:11.373050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b7ba7-82a9-46d6-8ff9-44e37456f4c7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:11.379626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd87b1f-04ce-4956-97ed-7e47d460fc12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:11.380652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd87b1f-04ce-4956-97ed-7e47d460fc12 service placement] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:11.381092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd87b1f-04ce-4956-97ed-7e47d460fc12 service placement] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:11.383842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd87b1f-04ce-4956-97ed-7e47d460fc12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:11.385709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2dd87b1f-04ce-4956-97ed-7e47d460fc12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:11.398816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd87b1f-04ce-4956-97ed-7e47d460fc12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:11.400404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd87b1f-04ce-4956-97ed-7e47d460fc12 service placement] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:11.417882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1192/2408] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:09:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:09:13.872285 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ddb9f-b456-4c52-a704-b30d7405a40e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:13.872747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ddb9f-b456-4c52-a704-b30d7405a40e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:13.872864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ddb9f-b456-4c52-a704-b30d7405a40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:13.891094 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6d3ddb9f-b456-4c52-a704-b30d7405a40e None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:13.892969 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6d3ddb9f-b456-4c52-a704-b30d7405a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:13.950220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1217/2409] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:09:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:09:13.973131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd87b1f-04ce-4956-97ed-7e47d460fc12 service placement] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:13.974639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2dd87b1f-04ce-4956-97ed-7e47d460fc12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:14.041774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd87b1f-04ce-4956-97ed-7e47d460fc12 service placement] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:14.046729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e3864-49f9-4ad3-82fa-40a62ce6d316 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:14.047365 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e3864-49f9-4ad3-82fa-40a62ce6d316 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:14.047684 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e3864-49f9-4ad3-82fa-40a62ce6d316 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:14.048041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e3864-49f9-4ad3-82fa-40a62ce6d316 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:14.056000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e3864-49f9-4ad3-82fa-40a62ce6d316 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:14.057565 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e3864-49f9-4ad3-82fa-40a62ce6d316 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:14.067016 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1193/2410] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:09:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:09:23.943647 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:24.063327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=77 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:27.204712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ddb9f-b456-4c52-a704-b30d7405a40e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:27.271771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ddb9f-b456-4c52-a704-b30d7405a40e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:27.276324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:27.277173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:27.277450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:27.277718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] PATH_INFO: `/v3/users/144831778810401d9c11b579b1ccad54` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:27.278905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=144831778810401d9c11b579b1ccad54)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:27.280830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:27.434726 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '144831778810401d9c11b579b1ccad54'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:09:27.441598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '144831778810401d9c11b579b1ccad54'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:09:27.523217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '144831778810401d9c11b579b1ccad54'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:09:27.523770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '144831778810401d9c11b579b1ccad54'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:09:27.524373 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '144831778810401d9c11b579b1ccad54'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:09:27.542989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1218/2411] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:09:27 2022] DELETE /identity/v3/users/144831778810401d9c11b579b1ccad54 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:09:27.547779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aba577-da9e-4bcf-83be-74a9ee895134 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:27.549884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aba577-da9e-4bcf-83be-74a9ee895134 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:27.549884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aba577-da9e-4bcf-83be-74a9ee895134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:27.565492 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-75aba577-da9e-4bcf-83be-74a9ee895134 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:27.566707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-75aba577-da9e-4bcf-83be-74a9ee895134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:27.626710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1194/2412] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:09:27.637928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:27.639376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:27.696243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8b9e5-c788-41db-ae47-78ad58cadf0b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:27.699935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a145c-06a4-491d-8b30-81203cc15f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:27.700569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a145c-06a4-491d-8b30-81203cc15f5c service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:27.700855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a145c-06a4-491d-8b30-81203cc15f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:27.701132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a145c-06a4-491d-8b30-81203cc15f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:27.710671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a145c-06a4-491d-8b30-81203cc15f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:27.712102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a145c-06a4-491d-8b30-81203cc15f5c service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:27.721485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1219/2413] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:09:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:09:28.102950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aba577-da9e-4bcf-83be-74a9ee895134 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:28.104601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-75aba577-da9e-4bcf-83be-74a9ee895134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:28.122342 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aba577-da9e-4bcf-83be-74a9ee895134 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:28.146226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:28.146226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.146226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.146226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] PATH_INFO: `/v3/projects/0df96ccaf9c848519ceda061b574228f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.147014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df96ccaf9c848519ceda061b574228f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:28.147014 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:28.256476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df96ccaf9c848519ceda061b574228f', 'request_id': 'req-e7886956-bd01-4d3f-a168-cb1218a926e9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:09:28.262233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df96ccaf9c848519ceda061b574228f', 'request_id': 'req-e7886956-bd01-4d3f-a168-cb1218a926e9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:09:28.329217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df96ccaf9c848519ceda061b574228f', 'request_id': 'req-e7886956-bd01-4d3f-a168-cb1218a926e9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:09:28.342274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] The token cache is being invalidate because project 0df96ccaf9c848519ceda061b574228f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:09:28.342586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:09:28.343195 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e7886956-bd01-4d3f-a168-cb1218a926e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:09:28.344716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1195/2414] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:09:28 2022] DELETE /identity/v3/projects/0df96ccaf9c848519ceda061b574228f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:09:28.350910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffb356-0c72-4dba-8204-79c1fdb70fcb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.351377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffb356-0c72-4dba-8204-79c1fdb70fcb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.351377 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffb356-0c72-4dba-8204-79c1fdb70fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.369497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7fffb356-0c72-4dba-8204-79c1fdb70fcb None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:28.370727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7fffb356-0c72-4dba-8204-79c1fdb70fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:28.387584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:28.429523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1220/2415] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:09:28.437218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023e76-7b98-4118-99dc-dc82f6f212d4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.437563 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023e76-7b98-4118-99dc-dc82f6f212d4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.437846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e023e76-7b98-4118-99dc-dc82f6f212d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.455762 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7e023e76-7b98-4118-99dc-dc82f6f212d4 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:28.456943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7e023e76-7b98-4118-99dc-dc82f6f212d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:28.464994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1196/2416] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:09:28.469153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffb356-0c72-4dba-8204-79c1fdb70fcb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:28.479511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffb356-0c72-4dba-8204-79c1fdb70fcb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:28.484493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552cb1a-165b-4cc3-9b0a-df4750225e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:28.485362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552cb1a-165b-4cc3-9b0a-df4750225e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.486049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552cb1a-165b-4cc3-9b0a-df4750225e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.486343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552cb1a-165b-4cc3-9b0a-df4750225e8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.487070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552cb1a-165b-4cc3-9b0a-df4750225e8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:09:28.487374 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552cb1a-165b-4cc3-9b0a-df4750225e8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:28.489062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552cb1a-165b-4cc3-9b0a-df4750225e8d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:28.501406 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1221/2417] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:09:28.505716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e023e76-7b98-4118-99dc-dc82f6f212d4 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:28.515408 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e023e76-7b98-4118-99dc-dc82f6f212d4 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:28.520157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ac27c-8cbb-4e85-927b-253081766ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:28.524252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ac27c-8cbb-4e85-927b-253081766ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.524252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ac27c-8cbb-4e85-927b-253081766ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.524252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ac27c-8cbb-4e85-927b-253081766ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.524252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ac27c-8cbb-4e85-927b-253081766ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:28.524252 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ac27c-8cbb-4e85-927b-253081766ba2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:28.605539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1197/2418] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:09:28 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:09:28.609614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552cb1a-165b-4cc3-9b0a-df4750225e8d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:28.620523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552cb1a-165b-4cc3-9b0a-df4750225e8d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:28.680290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6b7ed-1ccf-4472-9a70-e0aa471e3bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:28.680966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6b7ed-1ccf-4472-9a70-e0aa471e3bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.681249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6b7ed-1ccf-4472-9a70-e0aa471e3bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.681527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6b7ed-1ccf-4472-9a70-e0aa471e3bdc admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.682269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6b7ed-1ccf-4472-9a70-e0aa471e3bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:28.684199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6b7ed-1ccf-4472-9a70-e0aa471e3bdc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:28.759999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1222/2419] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:09:28 2022] POST /identity/v3/users => generated 283 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:09:28.764666 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ac27c-8cbb-4e85-927b-253081766ba2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:28.776553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ac27c-8cbb-4e85-927b-253081766ba2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:28.781414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a81e52-2eeb-4b43-9347-e43e684874ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:28.782208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a81e52-2eeb-4b43-9347-e43e684874ea admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.782558 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a81e52-2eeb-4b43-9347-e43e684874ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.782902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a81e52-2eeb-4b43-9347-e43e684874ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.783709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a81e52-2eeb-4b43-9347-e43e684874ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:28.785432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a81e52-2eeb-4b43-9347-e43e684874ea admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:28.790095 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450ec74-f3bf-4e88-aabc-fdf70f1723df None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.790287 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450ec74-f3bf-4e88-aabc-fdf70f1723df None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.790567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450ec74-f3bf-4e88-aabc-fdf70f1723df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.797454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1198/2420] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:09:28.800909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a81e52-2eeb-4b43-9347-e43e684874ea admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:28.808774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5450ec74-f3bf-4e88-aabc-fdf70f1723df None None] MFA Rules not processed for user `6f101dab08684c2483f67a9357efeae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:28.809955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5450ec74-f3bf-4e88-aabc-fdf70f1723df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:28.810784 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a81e52-2eeb-4b43-9347-e43e684874ea admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:28.815598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0421d76-48f8-40d4-a2cb-75cba9c0f9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:28.816245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0421d76-48f8-40d4-a2cb-75cba9c0f9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.816530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0421d76-48f8-40d4-a2cb-75cba9c0f9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.816807 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0421d76-48f8-40d4-a2cb-75cba9c0f9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.817715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0421d76-48f8-40d4-a2cb-75cba9c0f9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:28.819331 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0421d76-48f8-40d4-a2cb-75cba9c0f9c0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:28.829387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1199/2421] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:09:28.834015 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0421d76-48f8-40d4-a2cb-75cba9c0f9c0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:28.844738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0421d76-48f8-40d4-a2cb-75cba9c0f9c0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:28.850282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8c1c5-f6a5-4113-ba45-9fe8bf7ee13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:28.851185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8c1c5-f6a5-4113-ba45-9fe8bf7ee13a admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.851468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8c1c5-f6a5-4113-ba45-9fe8bf7ee13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.851741 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8c1c5-f6a5-4113-ba45-9fe8bf7ee13a admin admin] PATH_INFO: `/v3/projects/22f2ea9e505e4e05825cff48a1d2adc5/users/4f9b8fb375ae498a9a7bb136b06a27bf/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.866670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8c1c5-f6a5-4113-ba45-9fe8bf7ee13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f2ea9e505e4e05825cff48a1d2adc5, user_id=4f9b8fb375ae498a9a7bb136b06a27bf, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:28.867540 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1223/2422] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:09:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:09:28.870198 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8c1c5-f6a5-4113-ba45-9fe8bf7ee13a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:28.935479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1200/2423] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:09:28 2022] PUT /identity/v3/projects/22f2ea9e505e4e05825cff48a1d2adc5/users/4f9b8fb375ae498a9a7bb136b06a27bf/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:09:28.946955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc3eb-e434-451c-b5bb-9d75baf96d8f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:28.947244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc3eb-e434-451c-b5bb-9d75baf96d8f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:28.947550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc3eb-e434-451c-b5bb-9d75baf96d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:28.976105 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-55dcc3eb-e434-451c-b5bb-9d75baf96d8f None None] MFA Rules not processed for user `4f9b8fb375ae498a9a7bb136b06a27bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:28.977315 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-55dcc3eb-e434-451c-b5bb-9d75baf96d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:29.011195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:29.043873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1224/2424] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:09:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:09:29.055362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8c1c5-f6a5-4113-ba45-9fe8bf7ee13a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:29.057077 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3ce8c1c5-f6a5-4113-ba45-9fe8bf7ee13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:29.070599 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:29.116504 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8c1c5-f6a5-4113-ba45-9fe8bf7ee13a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:29.120219 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598257ef-8374-430b-8cd6-3310952a1893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:29.120858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598257ef-8374-430b-8cd6-3310952a1893 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:29.121139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598257ef-8374-430b-8cd6-3310952a1893 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:29.121411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598257ef-8374-430b-8cd6-3310952a1893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:29.130761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598257ef-8374-430b-8cd6-3310952a1893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:29.132188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598257ef-8374-430b-8cd6-3310952a1893 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:29.142825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1201/2425] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:09:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:09:39.124081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=12 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:39.256278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=22 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:44.626957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea59b67-cf79-4630-97ea-f9fc68fdcd6f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:44.627440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea59b67-cf79-4630-97ea-f9fc68fdcd6f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:44.627569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea59b67-cf79-4630-97ea-f9fc68fdcd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:44.650845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3ea59b67-cf79-4630-97ea-f9fc68fdcd6f None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:09:44.652049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3ea59b67-cf79-4630-97ea-f9fc68fdcd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:44.711844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1225/2426] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 13 09:09:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:09:44.724917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598257ef-8374-430b-8cd6-3310952a1893 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:44.726430 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-598257ef-8374-430b-8cd6-3310952a1893 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:44.800307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598257ef-8374-430b-8cd6-3310952a1893 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:44.805389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572be35b-015f-4219-9328-a3f620fa80a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:44.806052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572be35b-015f-4219-9328-a3f620fa80a1 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:44.806352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572be35b-015f-4219-9328-a3f620fa80a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:44.806623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572be35b-015f-4219-9328-a3f620fa80a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:44.815176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572be35b-015f-4219-9328-a3f620fa80a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:44.816777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572be35b-015f-4219-9328-a3f620fa80a1 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:44.825652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1202/2427] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:09:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:09:54.704156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=25 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:54.822461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=32 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:09:55.900628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea59b67-cf79-4630-97ea-f9fc68fdcd6f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:09:55.902105 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3ea59b67-cf79-4630-97ea-f9fc68fdcd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:55.962480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea59b67-cf79-4630-97ea-f9fc68fdcd6f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:09:55.966345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695e9ce-d370-4884-bbcb-c0f13e0981d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:09:55.967002 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695e9ce-d370-4884-bbcb-c0f13e0981d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:09:55.967275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695e9ce-d370-4884-bbcb-c0f13e0981d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:09:55.967552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695e9ce-d370-4884-bbcb-c0f13e0981d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:09:55.969940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6695e9ce-d370-4884-bbcb-c0f13e0981d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:09:56.036015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695e9ce-d370-4884-bbcb-c0f13e0981d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:09:56.037463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695e9ce-d370-4884-bbcb-c0f13e0981d9 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:09:56.046838 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1226/2428] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:09:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:10:06.042733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=50 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:10:09.910618 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572be35b-015f-4219-9328-a3f620fa80a1 service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:10:09.973657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572be35b-015f-4219-9328-a3f620fa80a1 service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:10:09.979253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:10:09.980338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:10:09.980613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:10:09.980882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] PATH_INFO: `/v3/users/4f9b8fb375ae498a9a7bb136b06a27bf` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:10:09.982134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9b8fb375ae498a9a7bb136b06a27bf)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:10:09.984041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:10:10.102713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9b8fb375ae498a9a7bb136b06a27bf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:10:10.110456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9b8fb375ae498a9a7bb136b06a27bf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:10:10.122776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9b8fb375ae498a9a7bb136b06a27bf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:10:10.168870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9b8fb375ae498a9a7bb136b06a27bf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:10:10.169540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9b8fb375ae498a9a7bb136b06a27bf'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:10:10.177272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1203/2429] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:10:09 2022] DELETE /identity/v3/users/4f9b8fb375ae498a9a7bb136b06a27bf => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:10:10.181584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1cc93-121c-4f43-a067-2c51dfd5832a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:10:10.182056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1cc93-121c-4f43-a067-2c51dfd5832a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:10:10.182598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1cc93-121c-4f43-a067-2c51dfd5832a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:10:10.200360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c3e1cc93-121c-4f43-a067-2c51dfd5832a None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:10:10.201547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c3e1cc93-121c-4f43-a067-2c51dfd5832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:10:10.263018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1227/2430] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:10:10.272524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:10:10.274268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:10:10.333382 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a84690-abde-4783-83aa-aa2f1088714c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:10:10.341596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475be30b-efa3-4535-bee0-1ba6bbd2cbd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:10:10.342280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be30b-efa3-4535-bee0-1ba6bbd2cbd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:10:10.342564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be30b-efa3-4535-bee0-1ba6bbd2cbd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:10:10.342840 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be30b-efa3-4535-bee0-1ba6bbd2cbd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:10:10.355903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475be30b-efa3-4535-bee0-1ba6bbd2cbd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:10:10.357319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475be30b-efa3-4535-bee0-1ba6bbd2cbd6 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:10:10.366641 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1204/2431] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:10:10.747235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1cc93-121c-4f43-a067-2c51dfd5832a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:10:10.748650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c3e1cc93-121c-4f43-a067-2c51dfd5832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:10:10.760682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1cc93-121c-4f43-a067-2c51dfd5832a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:10:10.766571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:10:10.767430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:10:10.767716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:10:10.767994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] PATH_INFO: `/v3/projects/22f2ea9e505e4e05825cff48a1d2adc5` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:10:10.769324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f2ea9e505e4e05825cff48a1d2adc5)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:10:10.771158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:10:10.872130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f2ea9e505e4e05825cff48a1d2adc5', 'request_id': 'req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:10:10.877594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f2ea9e505e4e05825cff48a1d2adc5', 'request_id': 'req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:10:10.883691 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f2ea9e505e4e05825cff48a1d2adc5', 'request_id': 'req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:10:10.945011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] The token cache is being invalidate because project 22f2ea9e505e4e05825cff48a1d2adc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:10:10.945298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:10:10.946082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f78d5e6a-a36e-4391-bff2-d68d5bf5e630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:10:10.947688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1228/2432] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:10:10 2022] DELETE /identity/v3/projects/22f2ea9e505e4e05825cff48a1d2adc5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:10:20.362436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:10:20.941710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:10:29.008469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade78b9-8029-4d6a-9fea-cfeb06b92093 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:10:29.009040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade78b9-8029-4d6a-9fea-cfeb06b92093 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:10:29.009040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade78b9-8029-4d6a-9fea-cfeb06b92093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:10:29.027251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fade78b9-8029-4d6a-9fea-cfeb06b92093 None None] MFA Rules not processed for user `d9bffb24cf26486a9a8947882b52ec98`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:10:29.028475 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fade78b9-8029-4d6a-9fea-cfeb06b92093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:10:29.081553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1205/2433] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:10:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:10:39.075148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=25 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:10:41.449975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ca24-2710-4571-8e12-bd10d9ea4989 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:10:41.450513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ca24-2710-4571-8e12-bd10d9ea4989 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:10:41.450622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ca24-2710-4571-8e12-bd10d9ea4989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:10:41.469114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8c05ca24-2710-4571-8e12-bd10d9ea4989 None None] MFA Rules not processed for user `1438ef91441f4493844a915ce12fcfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:10:41.477630 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8c05ca24-2710-4571-8e12-bd10d9ea4989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:10:41.531942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1229/2434] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:10:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:10:51.524205 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=25 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:10:58.921030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914f18-0cb1-4570-8882-7b9abf168052 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:10:58.921491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914f18-0cb1-4570-8882-7b9abf168052 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:10:58.921590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914f18-0cb1-4570-8882-7b9abf168052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:10:58.939255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e1914f18-0cb1-4570-8882-7b9abf168052 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:10:58.940442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e1914f18-0cb1-4570-8882-7b9abf168052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:10:58.994171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1206/2435] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 13 09:10:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:10:59.127104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05ca24-2710-4571-8e12-bd10d9ea4989 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:10:59.128563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8c05ca24-2710-4571-8e12-bd10d9ea4989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:10:59.203275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05ca24-2710-4571-8e12-bd10d9ea4989 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:10:59.208259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20668c-da6a-4ddd-94b4-465c9180467b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:10:59.208905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20668c-da6a-4ddd-94b4-465c9180467b service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:10:59.209190 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20668c-da6a-4ddd-94b4-465c9180467b service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:10:59.209467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20668c-da6a-4ddd-94b4-465c9180467b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:10:59.218324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20668c-da6a-4ddd-94b4-465c9180467b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:10:59.219859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20668c-da6a-4ddd-94b4-465c9180467b service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:10:59.227959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1230/2436] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:10:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:11:02.060854 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1914f18-0cb1-4570-8882-7b9abf168052 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:02.062903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e1914f18-0cb1-4570-8882-7b9abf168052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:02.133321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1914f18-0cb1-4570-8882-7b9abf168052 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:02.137919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc0af9-dd75-4cf7-8b01-18ad440048a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:02.143255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc0af9-dd75-4cf7-8b01-18ad440048a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:02.143593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc0af9-dd75-4cf7-8b01-18ad440048a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:02.144638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc0af9-dd75-4cf7-8b01-18ad440048a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:02.147046 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-36dc0af9-dd75-4cf7-8b01-18ad440048a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:02.331048 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc0af9-dd75-4cf7-8b01-18ad440048a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:02.332680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc0af9-dd75-4cf7-8b01-18ad440048a4 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:02.346493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1207/2437] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:08.175706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d880-a43d-44d6-9c34-a09847d587ea None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:08.176173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d880-a43d-44d6-9c34-a09847d587ea None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:08.176329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d880-a43d-44d6-9c34-a09847d587ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:08.195372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-67d4d880-a43d-44d6-9c34-a09847d587ea None None] MFA Rules not processed for user `35c7c83855084c188ab8a6dacb57a24b`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:08.195533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc0af9-dd75-4cf7-8b01-18ad440048a4 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:08.196942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-36dc0af9-dd75-4cf7-8b01-18ad440048a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:08.197433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-67d4d880-a43d-44d6-9c34-a09847d587ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:08.270388 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:08.272404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1231/2438] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:08.290343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc0af9-dd75-4cf7-8b01-18ad440048a4 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:08.296586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:08.297502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:08.297781 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:08.298073 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] PATH_INFO: `/v3/users/1ac8c3e3e99e412bb4a1f7b1506abce2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:08.299265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac8c3e3e99e412bb4a1f7b1506abce2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:08.301175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:08.505462 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac8c3e3e99e412bb4a1f7b1506abce2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:08.516744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac8c3e3e99e412bb4a1f7b1506abce2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:08.535469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac8c3e3e99e412bb4a1f7b1506abce2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:08.625588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac8c3e3e99e412bb4a1f7b1506abce2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:08.626289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac8c3e3e99e412bb4a1f7b1506abce2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:08.634850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1208/2439] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:11:08 2022] DELETE /identity/v3/users/1ac8c3e3e99e412bb4a1f7b1506abce2 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:08.896139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4d880-a43d-44d6-9c34-a09847d587ea None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:08.908613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4d880-a43d-44d6-9c34-a09847d587ea None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:08.971104 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:08.971993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:08.972279 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:08.972557 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] PATH_INFO: `/v3/projects/be3ff54edaf7439bab9ca81656e94c70` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:08.973935 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3ff54edaf7439bab9ca81656e94c70)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:08.975910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:08.982595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:08.984166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:09.062693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02856bc-6cd5-4271-83fc-bbd7694db22c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:09.067394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc578948-4ac2-4ba0-a3cc-7a409903d6a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:09.068070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578948-4ac2-4ba0-a3cc-7a409903d6a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:09.068444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578948-4ac2-4ba0-a3cc-7a409903d6a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:09.068811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578948-4ac2-4ba0-a3cc-7a409903d6a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:09.071621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fc578948-4ac2-4ba0-a3cc-7a409903d6a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:09.085757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:09.096134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3ff54edaf7439bab9ca81656e94c70', 'request_id': 'req-7fcd2571-5492-4210-bb28-ad4eecdbdf20'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.103420 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3ff54edaf7439bab9ca81656e94c70', 'request_id': 'req-7fcd2571-5492-4210-bb28-ad4eecdbdf20'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.111293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3ff54edaf7439bab9ca81656e94c70', 'request_id': 'req-7fcd2571-5492-4210-bb28-ad4eecdbdf20'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.160499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc578948-4ac2-4ba0-a3cc-7a409903d6a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:09.161711 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc578948-4ac2-4ba0-a3cc-7a409903d6a9 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:09.173811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1209/2440] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:11:09.190770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] The token cache is being invalidate because project be3ff54edaf7439bab9ca81656e94c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:11:09.190770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.190770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.191350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1232/2441] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:11:08 2022] DELETE /identity/v3/projects/be3ff54edaf7439bab9ca81656e94c70 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:09.195274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc578948-4ac2-4ba0-a3cc-7a409903d6a9 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:09.196819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fc578948-4ac2-4ba0-a3cc-7a409903d6a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:09.220148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc578948-4ac2-4ba0-a3cc-7a409903d6a9 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:09.223980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:09.279438 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:09.280330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:09.280746 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:09.281259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] PATH_INFO: `/v3/users/342a6239810e40aa83db81985af2bfc2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:09.282512 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=342a6239810e40aa83db81985af2bfc2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:09.284446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:09.515643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342a6239810e40aa83db81985af2bfc2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.524823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342a6239810e40aa83db81985af2bfc2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.537207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342a6239810e40aa83db81985af2bfc2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.592310 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342a6239810e40aa83db81985af2bfc2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.592981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6cb63855-0d63-4023-8772-685896c36701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342a6239810e40aa83db81985af2bfc2'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.599792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1210/2442] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:11:09 2022] DELETE /identity/v3/users/342a6239810e40aa83db81985af2bfc2 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:09.815766 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:09.817229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:09.830694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd2571-5492-4210-bb28-ad4eecdbdf20 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:09.890985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:09.891847 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:09.892129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:09.892404 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] PATH_INFO: `/v3/projects/23ca6e44ed8f47fbbe0960fc3d65025f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:09.893682 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ca6e44ed8f47fbbe0960fc3d65025f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:09.895735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:09.990599 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ca6e44ed8f47fbbe0960fc3d65025f', 'request_id': 'req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:09.995828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ca6e44ed8f47fbbe0960fc3d65025f', 'request_id': 'req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:10.002396 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ca6e44ed8f47fbbe0960fc3d65025f', 'request_id': 'req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:10.047491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] The token cache is being invalidate because project 23ca6e44ed8f47fbbe0960fc3d65025f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:11:10.047773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:10.048382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d54ebcda-e2cc-4de4-8fed-e14fcbf1ddad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:10.049998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1233/2443] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:11:09 2022] DELETE /identity/v3/projects/23ca6e44ed8f47fbbe0960fc3d65025f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:10.055218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265f065-2a7b-4ea2-a8ed-997b99915158 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:10.055530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265f065-2a7b-4ea2-a8ed-997b99915158 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:10.055803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265f065-2a7b-4ea2-a8ed-997b99915158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:10.075337 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c265f065-2a7b-4ea2-a8ed-997b99915158 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:10.076541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c265f065-2a7b-4ea2-a8ed-997b99915158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:10.138675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1211/2444] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:10.146403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe7b80-6741-4c7a-90a5-8a94e2baef62 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:10.146686 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe7b80-6741-4c7a-90a5-8a94e2baef62 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:10.146956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe7b80-6741-4c7a-90a5-8a94e2baef62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:10.165995 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3dbe7b80-6741-4c7a-90a5-8a94e2baef62 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:10.167238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3dbe7b80-6741-4c7a-90a5-8a94e2baef62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:10.175792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1234/2445] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:10.179928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265f065-2a7b-4ea2-a8ed-997b99915158 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:10.190262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265f065-2a7b-4ea2-a8ed-997b99915158 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:10.194837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591da23-9e8a-45b9-9fde-d452961e2a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:10.195483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591da23-9e8a-45b9-9fde-d452961e2a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:10.195763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591da23-9e8a-45b9-9fde-d452961e2a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:10.196036 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591da23-9e8a-45b9-9fde-d452961e2a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:10.196751 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591da23-9e8a-45b9-9fde-d452961e2a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:11:10.197054 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591da23-9e8a-45b9-9fde-d452961e2a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:10.198892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591da23-9e8a-45b9-9fde-d452961e2a3f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:10.210721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1212/2446] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:11:10.214994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe7b80-6741-4c7a-90a5-8a94e2baef62 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:10.224176 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe7b80-6741-4c7a-90a5-8a94e2baef62 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:10.228773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3659ce-1e14-4696-b9fc-fb2ac3faeae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:10.229415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3659ce-1e14-4696-b9fc-fb2ac3faeae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:10.229693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3659ce-1e14-4696-b9fc-fb2ac3faeae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:10.229967 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3659ce-1e14-4696-b9fc-fb2ac3faeae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:10.230675 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3659ce-1e14-4696-b9fc-fb2ac3faeae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:10.232556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3659ce-1e14-4696-b9fc-fb2ac3faeae6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:10.338993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1235/2447] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:11:10 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:10.339894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591da23-9e8a-45b9-9fde-d452961e2a3f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:10.350429 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591da23-9e8a-45b9-9fde-d452961e2a3f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:10.360735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:10.411338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb2cf4-2600-4e13-a107-65397a532972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:10.412012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb2cf4-2600-4e13-a107-65397a532972 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:10.412301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb2cf4-2600-4e13-a107-65397a532972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:10.412580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb2cf4-2600-4e13-a107-65397a532972 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:10.413280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb2cf4-2600-4e13-a107-65397a532972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:10.415224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb2cf4-2600-4e13-a107-65397a532972 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:10.534008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1213/2448] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:11:10 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:11:10.538197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3659ce-1e14-4696-b9fc-fb2ac3faeae6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:10.548941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3659ce-1e14-4696-b9fc-fb2ac3faeae6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:10.553581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af1731-97eb-4729-bb8a-4853b5a30aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:10.556069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af1731-97eb-4729-bb8a-4853b5a30aab admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:10.556069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af1731-97eb-4729-bb8a-4853b5a30aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:10.556069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af1731-97eb-4729-bb8a-4853b5a30aab admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:10.556069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af1731-97eb-4729-bb8a-4853b5a30aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:10.557077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af1731-97eb-4729-bb8a-4853b5a30aab admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:10.568298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1236/2449] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:10.572185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb2cf4-2600-4e13-a107-65397a532972 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:10.582207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb2cf4-2600-4e13-a107-65397a532972 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:10.586826 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979e1fc-2db9-4840-ac7e-c54080fde468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:10.587471 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979e1fc-2db9-4840-ac7e-c54080fde468 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:10.587748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979e1fc-2db9-4840-ac7e-c54080fde468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:10.588019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979e1fc-2db9-4840-ac7e-c54080fde468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:10.588730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979e1fc-2db9-4840-ac7e-c54080fde468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:10.590402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979e1fc-2db9-4840-ac7e-c54080fde468 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:10.600172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1214/2450] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:10.604216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af1731-97eb-4729-bb8a-4853b5a30aab admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:10.614623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af1731-97eb-4729-bb8a-4853b5a30aab admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:10.619147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68ccd1-fc28-45a1-8aff-00d1867453c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:10.620017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68ccd1-fc28-45a1-8aff-00d1867453c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:10.620303 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68ccd1-fc28-45a1-8aff-00d1867453c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:10.620582 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68ccd1-fc28-45a1-8aff-00d1867453c4 admin admin] PATH_INFO: `/v3/projects/6b5574138d6341268eae22b78af64765/users/0a9a75227440420fb3eab1685598f2dc/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:10.633000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:10.636245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68ccd1-fc28-45a1-8aff-00d1867453c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5574138d6341268eae22b78af64765, user_id=0a9a75227440420fb3eab1685598f2dc, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:10.638976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68ccd1-fc28-45a1-8aff-00d1867453c4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:10.722291 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1237/2451] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:10 2022] PUT /identity/v3/projects/6b5574138d6341268eae22b78af64765/users/0a9a75227440420fb3eab1685598f2dc/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:10.727384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9097a78-45d9-44ae-be8a-018768a6770f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:10.727773 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9097a78-45d9-44ae-be8a-018768a6770f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:10.728143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9097a78-45d9-44ae-be8a-018768a6770f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:10.747107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b9097a78-45d9-44ae-be8a-018768a6770f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:10.748355 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b9097a78-45d9-44ae-be8a-018768a6770f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:10.805847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1215/2452] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:10.815500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68ccd1-fc28-45a1-8aff-00d1867453c4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:10.817212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5c68ccd1-fc28-45a1-8aff-00d1867453c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:10.874470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68ccd1-fc28-45a1-8aff-00d1867453c4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:10.878010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b7ef2-7e69-4d3c-871f-dd4c46f2fa81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:10.878727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b7ef2-7e69-4d3c-871f-dd4c46f2fa81 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:10.879006 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b7ef2-7e69-4d3c-871f-dd4c46f2fa81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:10.879278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b7ef2-7e69-4d3c-871f-dd4c46f2fa81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:10.888751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b7ef2-7e69-4d3c-871f-dd4c46f2fa81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:10.890210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b7ef2-7e69-4d3c-871f-dd4c46f2fa81 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:10.899390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1238/2453] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:17.762240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd78d1-8563-470d-9f55-e4897c9c7102 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:17.762752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd78d1-8563-470d-9f55-e4897c9c7102 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:17.762855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd78d1-8563-470d-9f55-e4897c9c7102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:17.791749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fbcd78d1-8563-470d-9f55-e4897c9c7102 None None] MFA Rules not processed for user `0a9a75227440420fb3eab1685598f2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:17.793006 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fbcd78d1-8563-470d-9f55-e4897c9c7102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:17.864738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1216/2454] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:11:17.869418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b7ef2-7e69-4d3c-871f-dd4c46f2fa81 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:17.881581 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b7ef2-7e69-4d3c-871f-dd4c46f2fa81 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:17.888596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af76b60-20c1-4f01-a17c-1a4001a4f309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:17.889541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af76b60-20c1-4f01-a17c-1a4001a4f309 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:17.889827 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af76b60-20c1-4f01-a17c-1a4001a4f309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:17.890259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af76b60-20c1-4f01-a17c-1a4001a4f309 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:17.890972 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af76b60-20c1-4f01-a17c-1a4001a4f309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:17.892944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af76b60-20c1-4f01-a17c-1a4001a4f309 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:18.016577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1239/2455] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:11:17 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:18.020813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd78d1-8563-470d-9f55-e4897c9c7102 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:18.031841 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd78d1-8563-470d-9f55-e4897c9c7102 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:18.055226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:18.095220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c402c-d339-4f87-9d98-8a518f6f27a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:18.095894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c402c-d339-4f87-9d98-8a518f6f27a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:18.096172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c402c-d339-4f87-9d98-8a518f6f27a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:18.096446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c402c-d339-4f87-9d98-8a518f6f27a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:18.097134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c402c-d339-4f87-9d98-8a518f6f27a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:18.099067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c402c-d339-4f87-9d98-8a518f6f27a2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:18.187557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1217/2456] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:11:18 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:18.192826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af76b60-20c1-4f01-a17c-1a4001a4f309 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:18.204510 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af76b60-20c1-4f01-a17c-1a4001a4f309 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:18.209854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518a159-ff5f-4c7c-971b-2b5cf0c4d5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:18.210556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518a159-ff5f-4c7c-971b-2b5cf0c4d5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:18.210836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518a159-ff5f-4c7c-971b-2b5cf0c4d5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:18.211435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518a159-ff5f-4c7c-971b-2b5cf0c4d5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:18.211934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518a159-ff5f-4c7c-971b-2b5cf0c4d5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:18.213497 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518a159-ff5f-4c7c-971b-2b5cf0c4d5dd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:18.227226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1240/2457] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:18.234730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c402c-d339-4f87-9d98-8a518f6f27a2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:18.245877 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c402c-d339-4f87-9d98-8a518f6f27a2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:18.251278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4259a-24e5-43e8-8f61-6b6527f33299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:18.252168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4259a-24e5-43e8-8f61-6b6527f33299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:18.252444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4259a-24e5-43e8-8f61-6b6527f33299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:18.252714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4259a-24e5-43e8-8f61-6b6527f33299 admin admin] PATH_INFO: `/v3/projects/b487644732884eaabfc43581d1950b6e/users/9b0ebcbf81b74188b0aca48ec6b122aa/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:18.269927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4259a-24e5-43e8-8f61-6b6527f33299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b487644732884eaabfc43581d1950b6e, user_id=9b0ebcbf81b74188b0aca48ec6b122aa, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:18.272687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4259a-24e5-43e8-8f61-6b6527f33299 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:18.348579 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1218/2458] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:18 2022] PUT /identity/v3/projects/b487644732884eaabfc43581d1950b6e/users/9b0ebcbf81b74188b0aca48ec6b122aa/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:11:18.352578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518a159-ff5f-4c7c-971b-2b5cf0c4d5dd admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:18.363948 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518a159-ff5f-4c7c-971b-2b5cf0c4d5dd admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:18.439208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4259a-24e5-43e8-8f61-6b6527f33299 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:18.441212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-12e4259a-24e5-43e8-8f61-6b6527f33299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:18.462184 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6947f-c2e7-42f3-b613-c2e39ac01f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:18.462886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6947f-c2e7-42f3-b613-c2e39ac01f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:18.463168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6947f-c2e7-42f3-b613-c2e39ac01f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:18.463444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6947f-c2e7-42f3-b613-c2e39ac01f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:18.464199 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6947f-c2e7-42f3-b613-c2e39ac01f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:18.466627 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6947f-c2e7-42f3-b613-c2e39ac01f61 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:18.477307 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1241/2459] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:11:18.482524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6947f-c2e7-42f3-b613-c2e39ac01f61 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:18.494329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6947f-c2e7-42f3-b613-c2e39ac01f61 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:18.499382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3373b-4820-48d1-9adb-821ac0697138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:18.500265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3373b-4820-48d1-9adb-821ac0697138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:18.500556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3373b-4820-48d1-9adb-821ac0697138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:18.500829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3373b-4820-48d1-9adb-821ac0697138 admin admin] PATH_INFO: `/v3/projects/b487644732884eaabfc43581d1950b6e/users/9b0ebcbf81b74188b0aca48ec6b122aa/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:18.502819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3373b-4820-48d1-9adb-821ac0697138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b487644732884eaabfc43581d1950b6e, user_id=9b0ebcbf81b74188b0aca48ec6b122aa, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:18.505493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3373b-4820-48d1-9adb-821ac0697138 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:18.517140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1242/2460] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:18 2022] PUT /identity/v3/projects/b487644732884eaabfc43581d1950b6e/users/9b0ebcbf81b74188b0aca48ec6b122aa/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:11:18.525822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4259a-24e5-43e8-8f61-6b6527f33299 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:18.597651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3373b-4820-48d1-9adb-821ac0697138 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:18.599771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1cf3373b-4820-48d1-9adb-821ac0697138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:18.606495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01054a-b156-43d1-9d8d-d873e3202b1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:18.607825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01054a-b156-43d1-9d8d-d873e3202b1f service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:18.607825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01054a-b156-43d1-9d8d-d873e3202b1f service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:18.607825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01054a-b156-43d1-9d8d-d873e3202b1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:18.609722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0c01054a-b156-43d1-9d8d-d873e3202b1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:18.642095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:18.658203 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:18.671851 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3373b-4820-48d1-9adb-821ac0697138 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:18.677113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ca47e-3533-47b2-bbb4-eb7854ad2fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:18.677958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca47e-3533-47b2-bbb4-eb7854ad2fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:18.678276 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca47e-3533-47b2-bbb4-eb7854ad2fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:18.678606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca47e-3533-47b2-bbb4-eb7854ad2fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:18.680560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7a5ca47e-3533-47b2-bbb4-eb7854ad2fe3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:18.686479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01054a-b156-43d1-9d8d-d873e3202b1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:18.688532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01054a-b156-43d1-9d8d-d873e3202b1f service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:18.710926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1219/2461] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:11:18.756567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ca47e-3533-47b2-bbb4-eb7854ad2fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:18.758002 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ca47e-3533-47b2-bbb4-eb7854ad2fe3 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:18.774532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1243/2462] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:25.253224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd452ce-0cb1-43b7-961a-2af01f4691fc None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:25.256120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd452ce-0cb1-43b7-961a-2af01f4691fc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:25.256120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd452ce-0cb1-43b7-961a-2af01f4691fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:25.282369 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8cd452ce-0cb1-43b7-961a-2af01f4691fc None None] MFA Rules not processed for user `9b0ebcbf81b74188b0aca48ec6b122aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:25.283667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8cd452ce-0cb1-43b7-961a-2af01f4691fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:25.352045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1220/2463] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:25.366615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ca47e-3533-47b2-bbb4-eb7854ad2fe3 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:25.368009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7a5ca47e-3533-47b2-bbb4-eb7854ad2fe3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:25.383734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ca47e-3533-47b2-bbb4-eb7854ad2fe3 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:25.390020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c578dc-cffe-4392-bcf9-2d18830609bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:25.390690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c578dc-cffe-4392-bcf9-2d18830609bf service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:25.390971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c578dc-cffe-4392-bcf9-2d18830609bf service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:25.391256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c578dc-cffe-4392-bcf9-2d18830609bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:25.460051 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c578dc-cffe-4392-bcf9-2d18830609bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:25.461698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c578dc-cffe-4392-bcf9-2d18830609bf service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:25.472536 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1244/2464] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:27.907735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95a62-3e5b-4fa3-8a6c-70855a5db6b1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:27.908303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95a62-3e5b-4fa3-8a6c-70855a5db6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:27.908303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95a62-3e5b-4fa3-8a6c-70855a5db6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:27.926501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-74c95a62-3e5b-4fa3-8a6c-70855a5db6b1 None None] MFA Rules not processed for user `0a9a75227440420fb3eab1685598f2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:27.927717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-74c95a62-3e5b-4fa3-8a6c-70855a5db6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:27.939474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1221/2465] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:30.405559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c578dc-cffe-4392-bcf9-2d18830609bf service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:30.418436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c578dc-cffe-4392-bcf9-2d18830609bf service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:30.425817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64319175-4dd4-4382-a2ce-716683ce4508 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:30.426694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64319175-4dd4-4382-a2ce-716683ce4508 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:30.426986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64319175-4dd4-4382-a2ce-716683ce4508 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:30.427321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64319175-4dd4-4382-a2ce-716683ce4508 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:30.429603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-64319175-4dd4-4382-a2ce-716683ce4508 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:30.501685 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64319175-4dd4-4382-a2ce-716683ce4508 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:30.503458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64319175-4dd4-4382-a2ce-716683ce4508 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:30.516909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1245/2466] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:11:31.851871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95a62-3e5b-4fa3-8a6c-70855a5db6b1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:31.853567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-74c95a62-3e5b-4fa3-8a6c-70855a5db6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:31.922062 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95a62-3e5b-4fa3-8a6c-70855a5db6b1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:31.925750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b054c9e-5322-49c9-83d3-ae38f545ec8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:31.926428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054c9e-5322-49c9-83d3-ae38f545ec8c service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:31.926929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054c9e-5322-49c9-83d3-ae38f545ec8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:31.927202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054c9e-5322-49c9-83d3-ae38f545ec8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:31.929153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0b054c9e-5322-49c9-83d3-ae38f545ec8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:31.983411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:31.994828 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b054c9e-5322-49c9-83d3-ae38f545ec8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:31.996401 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b054c9e-5322-49c9-83d3-ae38f545ec8c service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:32.007707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1222/2467] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:32.207835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64319175-4dd4-4382-a2ce-716683ce4508 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:32.220559 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64319175-4dd4-4382-a2ce-716683ce4508 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:32.224506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767a714-b800-4fbb-8565-f43896db32f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:32.225204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767a714-b800-4fbb-8565-f43896db32f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:32.225487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767a714-b800-4fbb-8565-f43896db32f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:32.225821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767a714-b800-4fbb-8565-f43896db32f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:32.227735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3767a714-b800-4fbb-8565-f43896db32f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:32.240089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767a714-b800-4fbb-8565-f43896db32f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:32.241567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767a714-b800-4fbb-8565-f43896db32f0 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:32.253004 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1246/2468] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:35.872940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b054c9e-5322-49c9-83d3-ae38f545ec8c service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:35.874527 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0b054c9e-5322-49c9-83d3-ae38f545ec8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:35.889741 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b054c9e-5322-49c9-83d3-ae38f545ec8c service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:35.894397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:35.895498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:35.895754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:35.896021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] PATH_INFO: `/v3/users/d9bffb24cf26486a9a8947882b52ec98` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:35.897196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9bffb24cf26486a9a8947882b52ec98)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:35.899123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:36.021548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9bffb24cf26486a9a8947882b52ec98'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:36.028768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9bffb24cf26486a9a8947882b52ec98'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:36.040259 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9bffb24cf26486a9a8947882b52ec98'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:36.088563 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9bffb24cf26486a9a8947882b52ec98'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:36.090508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76e2d2b5-4603-4b50-8863-7934780859f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9bffb24cf26486a9a8947882b52ec98'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:36.103806 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1223/2469] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:11:35 2022] DELETE /identity/v3/users/d9bffb24cf26486a9a8947882b52ec98 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:36.439041 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767a714-b800-4fbb-8565-f43896db32f0 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:36.451898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767a714-b800-4fbb-8565-f43896db32f0 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:36.470361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:36.508842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:36.509950 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:36.510274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:36.510554 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] PATH_INFO: `/v3/projects/7afbdee349034d2182240f1f7230236f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:36.512380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7afbdee349034d2182240f1f7230236f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:36.514328 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:36.603797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7afbdee349034d2182240f1f7230236f', 'request_id': 'req-0f9c8080-cf46-404c-a148-729f6c891408'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:36.611561 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7afbdee349034d2182240f1f7230236f', 'request_id': 'req-0f9c8080-cf46-404c-a148-729f6c891408'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:36.619435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7afbdee349034d2182240f1f7230236f', 'request_id': 'req-0f9c8080-cf46-404c-a148-729f6c891408'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:36.688874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] The token cache is being invalidate because project 7afbdee349034d2182240f1f7230236f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:11:36.689077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:36.689675 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0f9c8080-cf46-404c-a148-729f6c891408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:36.691269 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1247/2470] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:11:36 2022] DELETE /identity/v3/projects/7afbdee349034d2182240f1f7230236f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:36.701938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441fbd9-636d-4992-bfdc-1670574feba5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:36.702319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441fbd9-636d-4992-bfdc-1670574feba5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:36.702601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441fbd9-636d-4992-bfdc-1670574feba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:36.720487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b441fbd9-636d-4992-bfdc-1670574feba5 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:36.721681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b441fbd9-636d-4992-bfdc-1670574feba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:36.782529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1224/2471] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:36.790482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481da10-ec01-449c-adb4-95107e7803bd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:36.790766 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481da10-ec01-449c-adb4-95107e7803bd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:36.791040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481da10-ec01-449c-adb4-95107e7803bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:36.810600 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0481da10-ec01-449c-adb4-95107e7803bd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:36.811792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0481da10-ec01-449c-adb4-95107e7803bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:36.819650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1248/2472] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:36.823695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441fbd9-636d-4992-bfdc-1670574feba5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:36.833198 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441fbd9-636d-4992-bfdc-1670574feba5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:36.837670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5688776-83dc-47b0-a940-ccd3dbab7531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:36.838347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5688776-83dc-47b0-a940-ccd3dbab7531 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:36.838621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5688776-83dc-47b0-a940-ccd3dbab7531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:36.838899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5688776-83dc-47b0-a940-ccd3dbab7531 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:36.839862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5688776-83dc-47b0-a940-ccd3dbab7531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:11:36.840164 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5688776-83dc-47b0-a940-ccd3dbab7531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:36.841736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5688776-83dc-47b0-a940-ccd3dbab7531 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:36.853116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1225/2473] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:11:36.857978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481da10-ec01-449c-adb4-95107e7803bd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:36.867994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481da10-ec01-449c-adb4-95107e7803bd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:36.873952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae3cd9-62ac-4469-8abf-44471c02fd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:36.874669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae3cd9-62ac-4469-8abf-44471c02fd6c admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:36.874976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae3cd9-62ac-4469-8abf-44471c02fd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:36.875232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae3cd9-62ac-4469-8abf-44471c02fd6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:36.875958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae3cd9-62ac-4469-8abf-44471c02fd6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:36.877932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae3cd9-62ac-4469-8abf-44471c02fd6c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:36.958893 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1249/2474] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:11:36 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:11:36.962886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5688776-83dc-47b0-a940-ccd3dbab7531 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:36.973137 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5688776-83dc-47b0-a940-ccd3dbab7531 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:37.020186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:37.030311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164eef43-9a59-4d23-a514-a2501f4be386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:37.030978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eef43-9a59-4d23-a514-a2501f4be386 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.033738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eef43-9a59-4d23-a514-a2501f4be386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.033738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eef43-9a59-4d23-a514-a2501f4be386 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.033738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164eef43-9a59-4d23-a514-a2501f4be386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:37.034280 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164eef43-9a59-4d23-a514-a2501f4be386 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:37.105412 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1226/2475] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:11:36 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:37.109513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae3cd9-62ac-4469-8abf-44471c02fd6c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:37.119735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae3cd9-62ac-4469-8abf-44471c02fd6c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:37.124992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab88831-12f0-40cd-9a19-c2de6d04e67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:37.125920 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab88831-12f0-40cd-9a19-c2de6d04e67f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.126249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab88831-12f0-40cd-9a19-c2de6d04e67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.126516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab88831-12f0-40cd-9a19-c2de6d04e67f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.127388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab88831-12f0-40cd-9a19-c2de6d04e67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:37.129329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab88831-12f0-40cd-9a19-c2de6d04e67f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:37.143479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1250/2476] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:37.147803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164eef43-9a59-4d23-a514-a2501f4be386 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:37.159455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164eef43-9a59-4d23-a514-a2501f4be386 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:37.164505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b6f4d-b927-40e6-8aa1-efb08030fd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:37.165165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b6f4d-b927-40e6-8aa1-efb08030fd9d admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.165443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b6f4d-b927-40e6-8aa1-efb08030fd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.165714 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b6f4d-b927-40e6-8aa1-efb08030fd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.166531 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b6f4d-b927-40e6-8aa1-efb08030fd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:37.168793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b6f4d-b927-40e6-8aa1-efb08030fd9d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:37.180003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1227/2477] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:37.183869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab88831-12f0-40cd-9a19-c2de6d04e67f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:37.201045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab88831-12f0-40cd-9a19-c2de6d04e67f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:37.210709 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163eb6c-d2f9-4107-b505-e2d743bc0a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:37.211603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163eb6c-d2f9-4107-b505-e2d743bc0a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.211881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163eb6c-d2f9-4107-b505-e2d743bc0a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.212179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163eb6c-d2f9-4107-b505-e2d743bc0a43 admin admin] PATH_INFO: `/v3/projects/70a3724e67d346c6bf19e052fce4d614/users/486fdfd0af964b7aaf410e66ef3ac78a/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.227612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163eb6c-d2f9-4107-b505-e2d743bc0a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a3724e67d346c6bf19e052fce4d614, user_id=486fdfd0af964b7aaf410e66ef3ac78a, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:37.230908 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163eb6c-d2f9-4107-b505-e2d743bc0a43 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:37.299466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1251/2478] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:37 2022] PUT /identity/v3/projects/70a3724e67d346c6bf19e052fce4d614/users/486fdfd0af964b7aaf410e66ef3ac78a/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:37.310593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa59df-8a4d-4217-90ae-b974883ce6d9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.310881 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa59df-8a4d-4217-90ae-b974883ce6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.311156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa59df-8a4d-4217-90ae-b974883ce6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.337942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4dfa59df-8a4d-4217-90ae-b974883ce6d9 None None] MFA Rules not processed for user `486fdfd0af964b7aaf410e66ef3ac78a`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:37.340099 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4dfa59df-8a4d-4217-90ae-b974883ce6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:37.406643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1228/2479] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:37.411139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163eb6c-d2f9-4107-b505-e2d743bc0a43 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:37.470810 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:37.484888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163eb6c-d2f9-4107-b505-e2d743bc0a43 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:37.489735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00bb46-52db-4c23-b0d0-30ca4d7badc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:37.490444 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00bb46-52db-4c23-b0d0-30ca4d7badc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.490887 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00bb46-52db-4c23-b0d0-30ca4d7badc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.491165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00bb46-52db-4c23-b0d0-30ca4d7badc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.491855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00bb46-52db-4c23-b0d0-30ca4d7badc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:37.493801 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00bb46-52db-4c23-b0d0-30ca4d7badc1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:37.598532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1252/2480] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:11:37 2022] POST /identity/v3/projects => generated 363 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:37.602435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa59df-8a4d-4217-90ae-b974883ce6d9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:37.613956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa59df-8a4d-4217-90ae-b974883ce6d9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:37.669162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251e4a1-e77e-46fc-b3cd-6621eedd7385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:37.669814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e4a1-e77e-46fc-b3cd-6621eedd7385 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.670138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e4a1-e77e-46fc-b3cd-6621eedd7385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.670421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e4a1-e77e-46fc-b3cd-6621eedd7385 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.671109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251e4a1-e77e-46fc-b3cd-6621eedd7385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:37.673011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251e4a1-e77e-46fc-b3cd-6621eedd7385 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:37.748786 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1229/2481] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:11:37 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:37.751040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00bb46-52db-4c23-b0d0-30ca4d7badc1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:37.761762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00bb46-52db-4c23-b0d0-30ca4d7badc1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:37.766498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4ade8-d353-424e-96b3-b6f697f38bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:37.767155 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4ade8-d353-424e-96b3-b6f697f38bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.767448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4ade8-d353-424e-96b3-b6f697f38bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.767728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4ade8-d353-424e-96b3-b6f697f38bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.768447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4ade8-d353-424e-96b3-b6f697f38bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:37.770068 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4ade8-d353-424e-96b3-b6f697f38bc6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:37.781009 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1253/2482] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:37.785191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251e4a1-e77e-46fc-b3cd-6621eedd7385 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:37.796340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251e4a1-e77e-46fc-b3cd-6621eedd7385 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:37.800832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163266c-ae2e-4522-b23f-c47a74a076ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:37.801695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163266c-ae2e-4522-b23f-c47a74a076ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.801972 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163266c-ae2e-4522-b23f-c47a74a076ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.802275 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163266c-ae2e-4522-b23f-c47a74a076ba admin admin] PATH_INFO: `/v3/projects/c236af51588044f5959f42758530569a/users/03b63396a0a1410bab68b3aaeba00d99/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.816762 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163266c-ae2e-4522-b23f-c47a74a076ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c236af51588044f5959f42758530569a, user_id=03b63396a0a1410bab68b3aaeba00d99, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:37.819428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163266c-ae2e-4522-b23f-c47a74a076ba admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:37.882122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1230/2483] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:37 2022] PUT /identity/v3/projects/c236af51588044f5959f42758530569a/users/03b63396a0a1410bab68b3aaeba00d99/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:37.886462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4ade8-d353-424e-96b3-b6f697f38bc6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:37.896556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4ade8-d353-424e-96b3-b6f697f38bc6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:37.959952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b82b15-9815-4d19-a695-b4c0f1b5d835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:37.960626 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b82b15-9815-4d19-a695-b4c0f1b5d835 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.960918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b82b15-9815-4d19-a695-b4c0f1b5d835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.961189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b82b15-9815-4d19-a695-b4c0f1b5d835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.961914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b82b15-9815-4d19-a695-b4c0f1b5d835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:37.963574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b82b15-9815-4d19-a695-b4c0f1b5d835 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:37.973804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1254/2484] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:37.978070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163266c-ae2e-4522-b23f-c47a74a076ba admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:37.989158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163266c-ae2e-4522-b23f-c47a74a076ba admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:37.993839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae875a3-00c8-4eee-adde-e54e3cb34193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:37.994748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae875a3-00c8-4eee-adde-e54e3cb34193 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:37.995030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae875a3-00c8-4eee-adde-e54e3cb34193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:37.995303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae875a3-00c8-4eee-adde-e54e3cb34193 admin admin] PATH_INFO: `/v3/projects/c236af51588044f5959f42758530569a/users/03b63396a0a1410bab68b3aaeba00d99/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:37.997334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae875a3-00c8-4eee-adde-e54e3cb34193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c236af51588044f5959f42758530569a, user_id=03b63396a0a1410bab68b3aaeba00d99, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:38.000047 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae875a3-00c8-4eee-adde-e54e3cb34193 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:38.010518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1231/2485] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:37 2022] PUT /identity/v3/projects/c236af51588044f5959f42758530569a/users/03b63396a0a1410bab68b3aaeba00d99/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:38.020780 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1b48-0601-4a1e-924c-d6e60b973102 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:38.021076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1b48-0601-4a1e-924c-d6e60b973102 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:38.021359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1b48-0601-4a1e-924c-d6e60b973102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:38.048237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5bea1b48-0601-4a1e-924c-d6e60b973102 None None] MFA Rules not processed for user `03b63396a0a1410bab68b3aaeba00d99`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:38.049431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5bea1b48-0601-4a1e-924c-d6e60b973102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:38.116417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1255/2486] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:38 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:38.127302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae875a3-00c8-4eee-adde-e54e3cb34193 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:38.128974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2ae875a3-00c8-4eee-adde-e54e3cb34193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:38.143590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:38.187933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae875a3-00c8-4eee-adde-e54e3cb34193 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:38.192018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c934a-f8d1-452f-9e5a-b15b554cc90f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:38.192709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c934a-f8d1-452f-9e5a-b15b554cc90f service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:38.193017 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c934a-f8d1-452f-9e5a-b15b554cc90f service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:38.194343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c934a-f8d1-452f-9e5a-b15b554cc90f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:38.205436 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c934a-f8d1-452f-9e5a-b15b554cc90f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:38.207056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c934a-f8d1-452f-9e5a-b15b554cc90f service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:38.219359 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1232/2487] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:39.105311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea1b48-0601-4a1e-924c-d6e60b973102 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:39.153803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:39.170833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea1b48-0601-4a1e-924c-d6e60b973102 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:39.175522 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:39.176387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:39.176673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:39.176953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] PATH_INFO: `/v3/users/486fdfd0af964b7aaf410e66ef3ac78a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:39.178214 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=486fdfd0af964b7aaf410e66ef3ac78a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:39.180965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:39.318699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '486fdfd0af964b7aaf410e66ef3ac78a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:39.325634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '486fdfd0af964b7aaf410e66ef3ac78a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:39.385680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '486fdfd0af964b7aaf410e66ef3ac78a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:39.386841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '486fdfd0af964b7aaf410e66ef3ac78a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:39.387447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '486fdfd0af964b7aaf410e66ef3ac78a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:39.406058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1256/2488] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:11:39 2022] DELETE /identity/v3/users/486fdfd0af964b7aaf410e66ef3ac78a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:39.410628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5956-9b87-4ade-8b3a-086d2f3e7965 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:39.411123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5956-9b87-4ade-8b3a-086d2f3e7965 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:39.411218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5956-9b87-4ade-8b3a-086d2f3e7965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:39.429420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3b7a5956-9b87-4ade-8b3a-086d2f3e7965 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:39.430602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3b7a5956-9b87-4ade-8b3a-086d2f3e7965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:39.489904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1233/2489] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:39.499710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:39.501154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:39.560734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381c12f-95b1-4e4e-9809-d06ac73f5c6f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:39.564362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f915a2f-e41c-4f97-acad-f1480e41d26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:39.564997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f915a2f-e41c-4f97-acad-f1480e41d26b service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:39.565274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f915a2f-e41c-4f97-acad-f1480e41d26b service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:39.565544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f915a2f-e41c-4f97-acad-f1480e41d26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:39.574751 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f915a2f-e41c-4f97-acad-f1480e41d26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:39.576180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f915a2f-e41c-4f97-acad-f1480e41d26b service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:39.585512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1257/2490] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:39.987234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a5956-9b87-4ade-8b3a-086d2f3e7965 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:39.988691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3b7a5956-9b87-4ade-8b3a-086d2f3e7965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:40.002930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a5956-9b87-4ade-8b3a-086d2f3e7965 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:40.007518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:40.011535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:40.011535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:40.011535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] PATH_INFO: `/v3/projects/70a3724e67d346c6bf19e052fce4d614` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:40.011535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a3724e67d346c6bf19e052fce4d614)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:40.013296 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:40.152949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a3724e67d346c6bf19e052fce4d614', 'request_id': 'req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:40.158538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a3724e67d346c6bf19e052fce4d614', 'request_id': 'req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:40.213588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a3724e67d346c6bf19e052fce4d614', 'request_id': 'req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:40.225321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] The token cache is being invalidate because project 70a3724e67d346c6bf19e052fce4d614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:11:40.225603 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:40.226258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:40.227784 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1234/2491] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:11:39 2022] DELETE /identity/v3/projects/70a3724e67d346c6bf19e052fce4d614 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:40.231922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f915a2f-e41c-4f97-acad-f1480e41d26b service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:40.241560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f915a2f-e41c-4f97-acad-f1480e41d26b service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:40.297982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:40.298872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:40.299153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:40.299426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] PATH_INFO: `/v3/users/03b63396a0a1410bab68b3aaeba00d99` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:40.300607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b63396a0a1410bab68b3aaeba00d99)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:40.302553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:40.307756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:40.479011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b63396a0a1410bab68b3aaeba00d99'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:40.486438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b63396a0a1410bab68b3aaeba00d99'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:40.550910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b63396a0a1410bab68b3aaeba00d99'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:40.551529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b63396a0a1410bab68b3aaeba00d99'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:40.552149 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-aba4303f-6e4d-4366-952e-1622bc1a55db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b63396a0a1410bab68b3aaeba00d99'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:40.570792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1258/2492] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:11:40 2022] DELETE /identity/v3/users/03b63396a0a1410bab68b3aaeba00d99 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:40.952948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:40.954394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:40.968117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafe9b6-ef29-43ff-8e20-38d164bf64c3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:41.004378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:41.028749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:41.029535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.029828 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.030141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] PATH_INFO: `/v3/projects/c236af51588044f5959f42758530569a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.031340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c236af51588044f5959f42758530569a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:41.033233 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:41.128735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c236af51588044f5959f42758530569a', 'request_id': 'req-6d47d131-9c95-4c03-b83f-cae370bc69fa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:41.134454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c236af51588044f5959f42758530569a', 'request_id': 'req-6d47d131-9c95-4c03-b83f-cae370bc69fa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:41.230649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c236af51588044f5959f42758530569a', 'request_id': 'req-6d47d131-9c95-4c03-b83f-cae370bc69fa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:41.242059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] The token cache is being invalidate because project c236af51588044f5959f42758530569a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:11:41.242364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:41.242998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6d47d131-9c95-4c03-b83f-cae370bc69fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:41.244581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1235/2493] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:11:40 2022] DELETE /identity/v3/projects/c236af51588044f5959f42758530569a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:41.249546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc886b0-5435-4111-b3de-5694320fe66f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.250011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc886b0-5435-4111-b3de-5694320fe66f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.250132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc886b0-5435-4111-b3de-5694320fe66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.268541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bcc886b0-5435-4111-b3de-5694320fe66f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:41.269741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bcc886b0-5435-4111-b3de-5694320fe66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:41.329403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1259/2494] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:41.338197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7de2ba-5bf5-4728-91c3-93c6e1a97c7f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.338477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7de2ba-5bf5-4728-91c3-93c6e1a97c7f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.342338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7de2ba-5bf5-4728-91c3-93c6e1a97c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.357498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8f7de2ba-5bf5-4728-91c3-93c6e1a97c7f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:41.358710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8f7de2ba-5bf5-4728-91c3-93c6e1a97c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:41.367466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1236/2495] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:41.371534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc886b0-5435-4111-b3de-5694320fe66f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:41.381151 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc886b0-5435-4111-b3de-5694320fe66f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:41.386046 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d25e59-8b46-464e-8b68-782648e15ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:41.387207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d25e59-8b46-464e-8b68-782648e15ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.387490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d25e59-8b46-464e-8b68-782648e15ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.387764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d25e59-8b46-464e-8b68-782648e15ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.388486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d25e59-8b46-464e-8b68-782648e15ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:11:41.388791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d25e59-8b46-464e-8b68-782648e15ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:41.390435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d25e59-8b46-464e-8b68-782648e15ea3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:41.405982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1260/2496] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:11:41.407449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7de2ba-5bf5-4728-91c3-93c6e1a97c7f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:41.416485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7de2ba-5bf5-4728-91c3-93c6e1a97c7f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:41.422024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75d20c-5acc-4b43-baee-eb2433b41c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:41.422691 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75d20c-5acc-4b43-baee-eb2433b41c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.422968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75d20c-5acc-4b43-baee-eb2433b41c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.423266 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75d20c-5acc-4b43-baee-eb2433b41c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.423958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75d20c-5acc-4b43-baee-eb2433b41c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:41.425843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75d20c-5acc-4b43-baee-eb2433b41c8c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:41.512723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1237/2497] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:11:41 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:41.516861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d25e59-8b46-464e-8b68-782648e15ea3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:41.527192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d25e59-8b46-464e-8b68-782648e15ea3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:41.584482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315b502-ccf4-42f7-a651-83cb5ffabf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:41.585064 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b502-ccf4-42f7-a651-83cb5ffabf73 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.585347 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b502-ccf4-42f7-a651-83cb5ffabf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.585625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b502-ccf4-42f7-a651-83cb5ffabf73 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.586334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315b502-ccf4-42f7-a651-83cb5ffabf73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:41.588248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315b502-ccf4-42f7-a651-83cb5ffabf73 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:41.645524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1261/2498] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:11:41 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:41.647120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75d20c-5acc-4b43-baee-eb2433b41c8c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:41.663055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75d20c-5acc-4b43-baee-eb2433b41c8c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:41.667790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961f30-d45f-440f-ae51-8c74c1c770c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:41.668452 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961f30-d45f-440f-ae51-8c74c1c770c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.668896 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961f30-d45f-440f-ae51-8c74c1c770c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.669192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961f30-d45f-440f-ae51-8c74c1c770c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.669920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4961f30-d45f-440f-ae51-8c74c1c770c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:41.671611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4961f30-d45f-440f-ae51-8c74c1c770c2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:41.682019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1238/2499] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:41.686817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315b502-ccf4-42f7-a651-83cb5ffabf73 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:41.692008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:41.697321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315b502-ccf4-42f7-a651-83cb5ffabf73 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:41.703186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc677f4-49d4-4e0b-b10e-723e0273c117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:41.703841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc677f4-49d4-4e0b-b10e-723e0273c117 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.704124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc677f4-49d4-4e0b-b10e-723e0273c117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.704397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc677f4-49d4-4e0b-b10e-723e0273c117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.705123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc677f4-49d4-4e0b-b10e-723e0273c117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:41.706807 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc677f4-49d4-4e0b-b10e-723e0273c117 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:41.717840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1262/2500] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:41.721763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961f30-d45f-440f-ae51-8c74c1c770c2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:41.731847 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961f30-d45f-440f-ae51-8c74c1c770c2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:41.736756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7968f5e-0fe3-4333-84be-47b4559c5701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:41.737625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7968f5e-0fe3-4333-84be-47b4559c5701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.737907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7968f5e-0fe3-4333-84be-47b4559c5701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.738218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7968f5e-0fe3-4333-84be-47b4559c5701 admin admin] PATH_INFO: `/v3/projects/e3d84955fc054d84bda904a088a3ce90/users/da6b25cdcbe84317a0bea4b3c5fa88f6/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.753186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7968f5e-0fe3-4333-84be-47b4559c5701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d84955fc054d84bda904a088a3ce90, user_id=da6b25cdcbe84317a0bea4b3c5fa88f6, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:41.755853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7968f5e-0fe3-4333-84be-47b4559c5701 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:41.814505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1239/2501] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:41 2022] PUT /identity/v3/projects/e3d84955fc054d84bda904a088a3ce90/users/da6b25cdcbe84317a0bea4b3c5fa88f6/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:11:41.825390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cdf64-c2d1-4242-89c9-3bb0530c0f87 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.832259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cdf64-c2d1-4242-89c9-3bb0530c0f87 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.832259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cdf64-c2d1-4242-89c9-3bb0530c0f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.851770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e66cdf64-c2d1-4242-89c9-3bb0530c0f87 None None] MFA Rules not processed for user `da6b25cdcbe84317a0bea4b3c5fa88f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:41.852898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e66cdf64-c2d1-4242-89c9-3bb0530c0f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:41.916549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1263/2502] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:11:41.921592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7968f5e-0fe3-4333-84be-47b4559c5701 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:41.983071 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7968f5e-0fe3-4333-84be-47b4559c5701 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:41.987766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04033a17-9ab4-4692-b4cd-9c77918f0c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:41.988416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04033a17-9ab4-4692-b4cd-9c77918f0c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:41.988699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04033a17-9ab4-4692-b4cd-9c77918f0c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:41.988975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04033a17-9ab4-4692-b4cd-9c77918f0c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:41.989674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04033a17-9ab4-4692-b4cd-9c77918f0c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:41.991597 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04033a17-9ab4-4692-b4cd-9c77918f0c25 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:42.000224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:42.089373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1240/2503] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:11:41 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:42.093598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cdf64-c2d1-4242-89c9-3bb0530c0f87 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:42.104235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cdf64-c2d1-4242-89c9-3bb0530c0f87 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:42.167936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d393e0-85ac-4012-a267-ada37a40e96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:42.169150 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d393e0-85ac-4012-a267-ada37a40e96a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:42.169435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d393e0-85ac-4012-a267-ada37a40e96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:42.169711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d393e0-85ac-4012-a267-ada37a40e96a admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:42.170451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d393e0-85ac-4012-a267-ada37a40e96a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:42.172379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d393e0-85ac-4012-a267-ada37a40e96a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:42.288519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1264/2504] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:11:42 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:42.295885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04033a17-9ab4-4692-b4cd-9c77918f0c25 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:42.306076 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04033a17-9ab4-4692-b4cd-9c77918f0c25 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:42.311191 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209006c2-83f6-4b86-b365-f8ce705ed11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:42.311839 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209006c2-83f6-4b86-b365-f8ce705ed11c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:42.312120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209006c2-83f6-4b86-b365-f8ce705ed11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:42.312393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209006c2-83f6-4b86-b365-f8ce705ed11c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:42.313112 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209006c2-83f6-4b86-b365-f8ce705ed11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:42.314736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209006c2-83f6-4b86-b365-f8ce705ed11c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:42.326476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1241/2505] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:42.330541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d393e0-85ac-4012-a267-ada37a40e96a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:42.340922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d393e0-85ac-4012-a267-ada37a40e96a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:42.345713 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873eccff-6d45-49a6-a80c-6c0872f5dac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:42.346708 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873eccff-6d45-49a6-a80c-6c0872f5dac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:42.347018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873eccff-6d45-49a6-a80c-6c0872f5dac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:42.347292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873eccff-6d45-49a6-a80c-6c0872f5dac7 admin admin] PATH_INFO: `/v3/projects/c41e97b5e80b48f688d0cc9a2a5d80ac/users/48799ac4fefc4943ac4da475ddfc6614/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:42.362556 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873eccff-6d45-49a6-a80c-6c0872f5dac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41e97b5e80b48f688d0cc9a2a5d80ac, user_id=48799ac4fefc4943ac4da475ddfc6614, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:42.365243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873eccff-6d45-49a6-a80c-6c0872f5dac7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:42.435774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1265/2506] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:42 2022] PUT /identity/v3/projects/c41e97b5e80b48f688d0cc9a2a5d80ac/users/48799ac4fefc4943ac4da475ddfc6614/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:42.436655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209006c2-83f6-4b86-b365-f8ce705ed11c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:42.446792 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209006c2-83f6-4b86-b365-f8ce705ed11c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:42.504693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f8beb-007e-4047-a28e-031aa2366e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:42.505356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f8beb-007e-4047-a28e-031aa2366e55 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:42.505640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f8beb-007e-4047-a28e-031aa2366e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:42.505914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f8beb-007e-4047-a28e-031aa2366e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:42.506667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f8beb-007e-4047-a28e-031aa2366e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:42.508253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f8beb-007e-4047-a28e-031aa2366e55 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:42.519031 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1242/2507] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:42.523032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873eccff-6d45-49a6-a80c-6c0872f5dac7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:42.533474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873eccff-6d45-49a6-a80c-6c0872f5dac7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:42.537993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edd87f-0e33-4ca4-a396-cbd935fba5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:42.538888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edd87f-0e33-4ca4-a396-cbd935fba5b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:42.539167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edd87f-0e33-4ca4-a396-cbd935fba5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:42.539437 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edd87f-0e33-4ca4-a396-cbd935fba5b4 admin admin] PATH_INFO: `/v3/projects/c41e97b5e80b48f688d0cc9a2a5d80ac/users/48799ac4fefc4943ac4da475ddfc6614/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:42.541395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edd87f-0e33-4ca4-a396-cbd935fba5b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41e97b5e80b48f688d0cc9a2a5d80ac, user_id=48799ac4fefc4943ac4da475ddfc6614, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:42.544455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edd87f-0e33-4ca4-a396-cbd935fba5b4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:42.554633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1266/2508] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:42 2022] PUT /identity/v3/projects/c41e97b5e80b48f688d0cc9a2a5d80ac/users/48799ac4fefc4943ac4da475ddfc6614/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:42.565642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32c434-63a4-4596-874d-b16d464aa54b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:42.565921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32c434-63a4-4596-874d-b16d464aa54b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:42.566267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32c434-63a4-4596-874d-b16d464aa54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:42.593650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9b32c434-63a4-4596-874d-b16d464aa54b None None] MFA Rules not processed for user `48799ac4fefc4943ac4da475ddfc6614`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:42.594885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9b32c434-63a4-4596-874d-b16d464aa54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:42.661811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1243/2509] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:42.673417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edd87f-0e33-4ca4-a396-cbd935fba5b4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:42.675098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c9edd87f-0e33-4ca4-a396-cbd935fba5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:42.711643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:42.732667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edd87f-0e33-4ca4-a396-cbd935fba5b4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:42.736326 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902824-a8d0-466d-8f1b-4a1c1666df7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:42.737015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902824-a8d0-466d-8f1b-4a1c1666df7e service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:42.737328 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902824-a8d0-466d-8f1b-4a1c1666df7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:42.737606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902824-a8d0-466d-8f1b-4a1c1666df7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:42.747991 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9902824-a8d0-466d-8f1b-4a1c1666df7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:42.749446 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9902824-a8d0-466d-8f1b-4a1c1666df7e service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:42.759626 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1267/2510] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:42.823092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32c434-63a4-4596-874d-b16d464aa54b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:42.885885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32c434-63a4-4596-874d-b16d464aa54b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:42.891165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:42.892057 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:42.892338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:42.892613 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] PATH_INFO: `/v3/users/da6b25cdcbe84317a0bea4b3c5fa88f6` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:42.893799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=da6b25cdcbe84317a0bea4b3c5fa88f6)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:42.895756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:42.897210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:43.078590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da6b25cdcbe84317a0bea4b3c5fa88f6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:43.085419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da6b25cdcbe84317a0bea4b3c5fa88f6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:43.097274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da6b25cdcbe84317a0bea4b3c5fa88f6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:43.168091 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da6b25cdcbe84317a0bea4b3c5fa88f6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:43.168697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da6b25cdcbe84317a0bea4b3c5fa88f6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:43.174995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1244/2511] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:11:42 2022] DELETE /identity/v3/users/da6b25cdcbe84317a0bea4b3c5fa88f6 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:43.179480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced6804-c899-42de-a8df-698b43564a88 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:43.182817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced6804-c899-42de-a8df-698b43564a88 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:43.182817 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced6804-c899-42de-a8df-698b43564a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:43.198206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fced6804-c899-42de-a8df-698b43564a88 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:43.200117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fced6804-c899-42de-a8df-698b43564a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:43.258447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1268/2512] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:43.269097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:43.270682 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:43.328887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791c60f-c2cd-4d27-af5a-4450580e67bc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:43.332562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c2203-c51e-4b49-a100-8f8a604e3ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:43.333207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c2203-c51e-4b49-a100-8f8a604e3ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:43.333489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c2203-c51e-4b49-a100-8f8a604e3ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:43.333765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c2203-c51e-4b49-a100-8f8a604e3ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:43.347084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c2203-c51e-4b49-a100-8f8a604e3ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:43.348432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c2203-c51e-4b49-a100-8f8a604e3ab6 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:43.359147 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1245/2513] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:43.809757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced6804-c899-42de-a8df-698b43564a88 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:43.811789 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fced6804-c899-42de-a8df-698b43564a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:43.824529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced6804-c899-42de-a8df-698b43564a88 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:43.829722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:43.830541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:43.830829 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:43.831106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] PATH_INFO: `/v3/projects/e3d84955fc054d84bda904a088a3ce90` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:43.832495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d84955fc054d84bda904a088a3ce90)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:43.834546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:43.929105 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d84955fc054d84bda904a088a3ce90', 'request_id': 'req-b47b8ff9-3e62-46ed-9bde-715560671a6d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:43.935456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d84955fc054d84bda904a088a3ce90', 'request_id': 'req-b47b8ff9-3e62-46ed-9bde-715560671a6d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:43.941903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d84955fc054d84bda904a088a3ce90', 'request_id': 'req-b47b8ff9-3e62-46ed-9bde-715560671a6d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:44.036225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] The token cache is being invalidate because project e3d84955fc054d84bda904a088a3ce90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:11:44.036511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:44.037124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:44.038700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1269/2514] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:11:43 2022] DELETE /identity/v3/projects/e3d84955fc054d84bda904a088a3ce90 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:44.042651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c2203-c51e-4b49-a100-8f8a604e3ab6 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:44.052852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c2203-c51e-4b49-a100-8f8a604e3ab6 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:44.108505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:44.109366 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:44.109793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:44.110236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] PATH_INFO: `/v3/users/48799ac4fefc4943ac4da475ddfc6614` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:44.111515 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] RBAC: Authorizing `identity:delete_user(user_id=48799ac4fefc4943ac4da475ddfc6614)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:44.113601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:44.139611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:44.353682 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48799ac4fefc4943ac4da475ddfc6614'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:44.360700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48799ac4fefc4943ac4da475ddfc6614'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:44.372862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48799ac4fefc4943ac4da475ddfc6614'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:44.476654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48799ac4fefc4943ac4da475ddfc6614'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:44.477230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48799ac4fefc4943ac4da475ddfc6614'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:44.483768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1246/2515] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:11:44 2022] DELETE /identity/v3/users/48799ac4fefc4943ac4da475ddfc6614 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:44.884511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:44.885914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:44.939476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:44.940927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:44.944322 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b8ff9-3e62-46ed-9bde-715560671a6d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:44.956649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:44.998942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1fe6-571b-401e-8d39-a3d4294f0728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:44.999501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1fe6-571b-401e-8d39-a3d4294f0728 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:44.999782 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1fe6-571b-401e-8d39-a3d4294f0728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.000056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1fe6-571b-401e-8d39-a3d4294f0728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.001879 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f4cb1fe6-571b-401e-8d39-a3d4294f0728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:45.011529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb1fe6-571b-401e-8d39-a3d4294f0728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:45.013634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb1fe6-571b-401e-8d39-a3d4294f0728 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:45.023163 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1270/2516] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:45.030521 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ad76d-2b0a-486c-9714-6be4fe77f0da admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:45.035156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:45.036127 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.036405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.036675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] PATH_INFO: `/v3/projects/c41e97b5e80b48f688d0cc9a2a5d80ac` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.037881 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c41e97b5e80b48f688d0cc9a2a5d80ac)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:45.039817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:45.176827 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c41e97b5e80b48f688d0cc9a2a5d80ac', 'request_id': 'req-a3de0e56-fd49-4c33-81ef-0f832d0c6568'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:45.182785 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c41e97b5e80b48f688d0cc9a2a5d80ac', 'request_id': 'req-a3de0e56-fd49-4c33-81ef-0f832d0c6568'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:45.265460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c41e97b5e80b48f688d0cc9a2a5d80ac', 'request_id': 'req-a3de0e56-fd49-4c33-81ef-0f832d0c6568'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:45.278149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] The token cache is being invalidate because project c41e97b5e80b48f688d0cc9a2a5d80ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:11:45.278594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:45.279186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a3de0e56-fd49-4c33-81ef-0f832d0c6568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:45.280819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1247/2517] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:11:44 2022] DELETE /identity/v3/projects/c41e97b5e80b48f688d0cc9a2a5d80ac => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:11:45.285577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd41bd5-ccee-4a53-b98f-b23aaaf6bdab None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.285944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd41bd5-ccee-4a53-b98f-b23aaaf6bdab None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.286360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd41bd5-ccee-4a53-b98f-b23aaaf6bdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.305466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bcd41bd5-ccee-4a53-b98f-b23aaaf6bdab None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:45.306711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bcd41bd5-ccee-4a53-b98f-b23aaaf6bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:45.369481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1271/2518] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:45.379140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf978ff7-03bd-45fa-a3c9-59383c3610ad None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.379425 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf978ff7-03bd-45fa-a3c9-59383c3610ad None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.379730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf978ff7-03bd-45fa-a3c9-59383c3610ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.398685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-cf978ff7-03bd-45fa-a3c9-59383c3610ad None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:45.399900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cf978ff7-03bd-45fa-a3c9-59383c3610ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:45.412004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1248/2519] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:45.416271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd41bd5-ccee-4a53-b98f-b23aaaf6bdab None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:45.425877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd41bd5-ccee-4a53-b98f-b23aaaf6bdab None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:45.430529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d65ae-8926-46f3-bcd5-1f701bf94eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:45.431181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d65ae-8926-46f3-bcd5-1f701bf94eaf admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.431466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d65ae-8926-46f3-bcd5-1f701bf94eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.431748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d65ae-8926-46f3-bcd5-1f701bf94eaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.432467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d65ae-8926-46f3-bcd5-1f701bf94eaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:11:45.432806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d65ae-8926-46f3-bcd5-1f701bf94eaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:45.435365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d65ae-8926-46f3-bcd5-1f701bf94eaf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:45.446924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1272/2520] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:11:45.452246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf978ff7-03bd-45fa-a3c9-59383c3610ad None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:45.461589 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf978ff7-03bd-45fa-a3c9-59383c3610ad None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:45.466533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a41394-95f9-47e4-a4a9-4f05f11882a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:45.467371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a41394-95f9-47e4-a4a9-4f05f11882a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.467713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a41394-95f9-47e4-a4a9-4f05f11882a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.468168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a41394-95f9-47e4-a4a9-4f05f11882a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.469421 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a41394-95f9-47e4-a4a9-4f05f11882a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:45.471481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a41394-95f9-47e4-a4a9-4f05f11882a3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:45.557986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1249/2521] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:11:45 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:45.562787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d65ae-8926-46f3-bcd5-1f701bf94eaf admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:45.573074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d65ae-8926-46f3-bcd5-1f701bf94eaf admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:45.634316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e165de7-ddfe-44fa-80c4-523b072bc95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:45.634979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e165de7-ddfe-44fa-80c4-523b072bc95d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.635259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e165de7-ddfe-44fa-80c4-523b072bc95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.635532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e165de7-ddfe-44fa-80c4-523b072bc95d admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.636252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e165de7-ddfe-44fa-80c4-523b072bc95d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:45.638241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e165de7-ddfe-44fa-80c4-523b072bc95d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:45.732770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1273/2522] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:11:45 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:45.733400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a41394-95f9-47e4-a4a9-4f05f11882a3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:45.744928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e165de7-ddfe-44fa-80c4-523b072bc95d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:45.745295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a41394-95f9-47e4-a4a9-4f05f11882a3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:45.746958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:45.748144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9e165de7-ddfe-44fa-80c4-523b072bc95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:45.753757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f4db3-222f-4c4c-8edb-a0700e2b757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:45.755796 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f4db3-222f-4c4c-8edb-a0700e2b757f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.756146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f4db3-222f-4c4c-8edb-a0700e2b757f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.756499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f4db3-222f-4c4c-8edb-a0700e2b757f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.757297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f4db3-222f-4c4c-8edb-a0700e2b757f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:45.759240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f4db3-222f-4c4c-8edb-a0700e2b757f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:45.784143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1250/2523] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:45.785197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f4db3-222f-4c4c-8edb-a0700e2b757f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:45.801783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f4db3-222f-4c4c-8edb-a0700e2b757f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:45.803803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:45.807970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482067-c252-4dba-8203-c7f896cf4867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:45.812964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482067-c252-4dba-8203-c7f896cf4867 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.812964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482067-c252-4dba-8203-c7f896cf4867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.812964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482067-c252-4dba-8203-c7f896cf4867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.812964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29482067-c252-4dba-8203-c7f896cf4867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:45.812964 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29482067-c252-4dba-8203-c7f896cf4867 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:45.826874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1251/2524] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:45.831298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482067-c252-4dba-8203-c7f896cf4867 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:45.840070 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e165de7-ddfe-44fa-80c4-523b072bc95d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:45.841867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482067-c252-4dba-8203-c7f896cf4867 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:45.845924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce74951-7f2d-4bac-90c4-e22bbd73976c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:45.846775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce74951-7f2d-4bac-90c4-e22bbd73976c service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.847059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce74951-7f2d-4bac-90c4-e22bbd73976c service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.847439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce74951-7f2d-4bac-90c4-e22bbd73976c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.848610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7feee9-dc9a-4693-9f5b-6cbbbe07d1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:45.849501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7feee9-dc9a-4693-9f5b-6cbbbe07d1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.849624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5ce74951-7f2d-4bac-90c4-e22bbd73976c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:45.850070 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7feee9-dc9a-4693-9f5b-6cbbbe07d1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.850363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7feee9-dc9a-4693-9f5b-6cbbbe07d1c9 admin admin] PATH_INFO: `/v3/projects/787f62f60be24ce49921e5017d48d768/users/903d2a5c0d094e469e9433451bd53d23/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:45.864504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce74951-7f2d-4bac-90c4-e22bbd73976c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:45.866245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce74951-7f2d-4bac-90c4-e22bbd73976c service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:45.867389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7feee9-dc9a-4693-9f5b-6cbbbe07d1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=787f62f60be24ce49921e5017d48d768, user_id=903d2a5c0d094e469e9433451bd53d23, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:45.870423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7feee9-dc9a-4693-9f5b-6cbbbe07d1c9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:45.879913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1274/2525] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:11:45.979759 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1252/2526] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:45 2022] PUT /identity/v3/projects/787f62f60be24ce49921e5017d48d768/users/903d2a5c0d094e469e9433451bd53d23/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:45.990862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46b03-09c4-4daa-9510-a5d788f54175 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:45.991331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46b03-09c4-4daa-9510-a5d788f54175 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:45.991427 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46b03-09c4-4daa-9510-a5d788f54175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:46.018467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-12a46b03-09c4-4daa-9510-a5d788f54175 None None] MFA Rules not processed for user `903d2a5c0d094e469e9433451bd53d23`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:46.019711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-12a46b03-09c4-4daa-9510-a5d788f54175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:46.084498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1275/2527] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:46.089046 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7feee9-dc9a-4693-9f5b-6cbbbe07d1c9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:46.150522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7feee9-dc9a-4693-9f5b-6cbbbe07d1c9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:46.155704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e071c76-8fe7-4b36-92a4-6c39986ee46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:46.156295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e071c76-8fe7-4b36-92a4-6c39986ee46d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:46.156572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e071c76-8fe7-4b36-92a4-6c39986ee46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:46.156838 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e071c76-8fe7-4b36-92a4-6c39986ee46d admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:46.157524 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e071c76-8fe7-4b36-92a4-6c39986ee46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:46.159427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e071c76-8fe7-4b36-92a4-6c39986ee46d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:46.228130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1253/2528] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:11:46 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:46.232814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a46b03-09c4-4daa-9510-a5d788f54175 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:46.243658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a46b03-09c4-4daa-9510-a5d788f54175 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:46.308451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f817a-7592-4696-93b1-7c30b077c24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:46.310523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f817a-7592-4696-93b1-7c30b077c24f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:46.310806 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f817a-7592-4696-93b1-7c30b077c24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:46.311121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f817a-7592-4696-93b1-7c30b077c24f admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:46.311813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f817a-7592-4696-93b1-7c30b077c24f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:46.313784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f817a-7592-4696-93b1-7c30b077c24f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:46.389067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1276/2529] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:11:46 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:46.390228 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e071c76-8fe7-4b36-92a4-6c39986ee46d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:46.400683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e071c76-8fe7-4b36-92a4-6c39986ee46d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:46.406052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a07ce-f858-4d87-bd61-4d5c03b79e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:46.406701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a07ce-f858-4d87-bd61-4d5c03b79e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:46.406978 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a07ce-f858-4d87-bd61-4d5c03b79e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:46.407289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a07ce-f858-4d87-bd61-4d5c03b79e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:46.408054 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a07ce-f858-4d87-bd61-4d5c03b79e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:46.409650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a07ce-f858-4d87-bd61-4d5c03b79e32 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:46.421074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1254/2530] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:11:46.425069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f817a-7592-4696-93b1-7c30b077c24f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:46.435853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f817a-7592-4696-93b1-7c30b077c24f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:46.440696 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd54675-08a2-4c9f-9f4d-b7c3c09993ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:46.441353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd54675-08a2-4c9f-9f4d-b7c3c09993ba admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:46.441688 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd54675-08a2-4c9f-9f4d-b7c3c09993ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:46.441991 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd54675-08a2-4c9f-9f4d-b7c3c09993ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:46.442754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd54675-08a2-4c9f-9f4d-b7c3c09993ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:46.444359 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd54675-08a2-4c9f-9f4d-b7c3c09993ba admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:46.454652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1277/2531] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:46.458616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a07ce-f858-4d87-bd61-4d5c03b79e32 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:46.468682 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a07ce-f858-4d87-bd61-4d5c03b79e32 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:46.473458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c37fb-bf63-4e93-9182-fdfa4e86ad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:46.474363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c37fb-bf63-4e93-9182-fdfa4e86ad36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:46.474647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c37fb-bf63-4e93-9182-fdfa4e86ad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:46.474923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c37fb-bf63-4e93-9182-fdfa4e86ad36 admin admin] PATH_INFO: `/v3/projects/551609a93a4149c8aab15ebfc281f3e1/users/ee64dea625f3445ba1c322edfed33ed3/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:46.491731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c37fb-bf63-4e93-9182-fdfa4e86ad36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=551609a93a4149c8aab15ebfc281f3e1, user_id=ee64dea625f3445ba1c322edfed33ed3, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:46.494654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c37fb-bf63-4e93-9182-fdfa4e86ad36 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:46.580083 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1255/2532] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:46 2022] PUT /identity/v3/projects/551609a93a4149c8aab15ebfc281f3e1/users/ee64dea625f3445ba1c322edfed33ed3/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:46.591002 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6927e-cb8f-44de-9bdd-67da07abb524 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:46.591292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6927e-cb8f-44de-9bdd-67da07abb524 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:46.591579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6927e-cb8f-44de-9bdd-67da07abb524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:46.620109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b3a6927e-cb8f-44de-9bdd-67da07abb524 None None] MFA Rules not processed for user `ee64dea625f3445ba1c322edfed33ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:46.622235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b3a6927e-cb8f-44de-9bdd-67da07abb524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:46.643336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:46.687692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1278/2533] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:46.692110 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c37fb-bf63-4e93-9182-fdfa4e86ad36 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:46.753900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c37fb-bf63-4e93-9182-fdfa4e86ad36 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:46.759055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc491e7-ef7c-4f73-9d71-5f794587f846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:46.759713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc491e7-ef7c-4f73-9d71-5f794587f846 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:46.759998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc491e7-ef7c-4f73-9d71-5f794587f846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:46.760274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc491e7-ef7c-4f73-9d71-5f794587f846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:46.761049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc491e7-ef7c-4f73-9d71-5f794587f846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:46.763114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc491e7-ef7c-4f73-9d71-5f794587f846 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:46.773333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:46.902011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1256/2534] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:11:46 2022] POST /identity/v3/projects => generated 377 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:46.905901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6927e-cb8f-44de-9bdd-67da07abb524 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:46.917034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6927e-cb8f-44de-9bdd-67da07abb524 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:46.976872 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994091ce-afff-4092-9d76-9091f339a26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:46.977548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994091ce-afff-4092-9d76-9091f339a26b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:46.977832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994091ce-afff-4092-9d76-9091f339a26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:46.978141 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994091ce-afff-4092-9d76-9091f339a26b admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:46.978852 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994091ce-afff-4092-9d76-9091f339a26b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:46.983255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994091ce-afff-4092-9d76-9091f339a26b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:47.082802 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1279/2535] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:11:46 2022] POST /identity/v3/users => generated 295 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:11:47.089725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc491e7-ef7c-4f73-9d71-5f794587f846 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:47.100967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc491e7-ef7c-4f73-9d71-5f794587f846 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:47.106283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0ec4c-5be2-47c5-a838-58da7983d51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:47.109916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ec4c-5be2-47c5-a838-58da7983d51b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:47.109916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ec4c-5be2-47c5-a838-58da7983d51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:47.109916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ec4c-5be2-47c5-a838-58da7983d51b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:47.109916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0ec4c-5be2-47c5-a838-58da7983d51b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:47.109916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0ec4c-5be2-47c5-a838-58da7983d51b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:47.120307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1257/2536] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:47.124505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994091ce-afff-4092-9d76-9091f339a26b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:47.135355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994091ce-afff-4092-9d76-9091f339a26b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:47.140808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6c19d-ace0-4ba2-aed2-596166179447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:47.141680 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6c19d-ace0-4ba2-aed2-596166179447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:47.141958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6c19d-ace0-4ba2-aed2-596166179447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:47.142265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6c19d-ace0-4ba2-aed2-596166179447 admin admin] PATH_INFO: `/v3/projects/51c6348829b74684836045fc5c61fee4/users/b12655a37dab412591ab5e4b86138657/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:47.158171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6c19d-ace0-4ba2-aed2-596166179447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c6348829b74684836045fc5c61fee4, user_id=b12655a37dab412591ab5e4b86138657, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:47.160906 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6c19d-ace0-4ba2-aed2-596166179447 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:47.216730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1280/2537] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:47 2022] PUT /identity/v3/projects/51c6348829b74684836045fc5c61fee4/users/b12655a37dab412591ab5e4b86138657/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:47.220705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0ec4c-5be2-47c5-a838-58da7983d51b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:47.231372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0ec4c-5be2-47c5-a838-58da7983d51b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:47.287800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4f41f-96ac-4db5-8543-c48a7982fb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:47.288465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4f41f-96ac-4db5-8543-c48a7982fb31 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:47.288745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4f41f-96ac-4db5-8543-c48a7982fb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:47.289018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4f41f-96ac-4db5-8543-c48a7982fb31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:47.289733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4f41f-96ac-4db5-8543-c48a7982fb31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:47.291363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4f41f-96ac-4db5-8543-c48a7982fb31 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:47.302189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1258/2538] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:47.306313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6c19d-ace0-4ba2-aed2-596166179447 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:47.317290 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6c19d-ace0-4ba2-aed2-596166179447 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:47.321982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1b24f-a95f-4398-bbce-feb4c35695d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:47.322886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1b24f-a95f-4398-bbce-feb4c35695d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:47.323168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1b24f-a95f-4398-bbce-feb4c35695d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:47.323443 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1b24f-a95f-4398-bbce-feb4c35695d1 admin admin] PATH_INFO: `/v3/projects/51c6348829b74684836045fc5c61fee4/users/b12655a37dab412591ab5e4b86138657/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:47.325808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1b24f-a95f-4398-bbce-feb4c35695d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c6348829b74684836045fc5c61fee4, user_id=b12655a37dab412591ab5e4b86138657, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:47.328531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1b24f-a95f-4398-bbce-feb4c35695d1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:47.339623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1281/2539] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:47 2022] PUT /identity/v3/projects/51c6348829b74684836045fc5c61fee4/users/b12655a37dab412591ab5e4b86138657/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:47.350883 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81290281-5c4a-457a-9f25-65f10e3583ae None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:47.351182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81290281-5c4a-457a-9f25-65f10e3583ae None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:47.351458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81290281-5c4a-457a-9f25-65f10e3583ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:47.382588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-81290281-5c4a-457a-9f25-65f10e3583ae None None] MFA Rules not processed for user `b12655a37dab412591ab5e4b86138657`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:47.382588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-81290281-5c4a-457a-9f25-65f10e3583ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:47.451192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1259/2540] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:11:47.462319 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1b24f-a95f-4398-bbce-feb4c35695d1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:47.464038 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-51a1b24f-a95f-4398-bbce-feb4c35695d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:47.521452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1b24f-a95f-4398-bbce-feb4c35695d1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:47.525025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c33987-b0bf-44c1-86e9-c3037408d850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:47.525662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33987-b0bf-44c1-86e9-c3037408d850 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:47.525946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33987-b0bf-44c1-86e9-c3037408d850 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:47.526253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33987-b0bf-44c1-86e9-c3037408d850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:47.535964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c33987-b0bf-44c1-86e9-c3037408d850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:47.537391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c33987-b0bf-44c1-86e9-c3037408d850 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:47.547501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1282/2541] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:47.638934 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81290281-5c4a-457a-9f25-65f10e3583ae None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:47.640330 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-81290281-5c4a-457a-9f25-65f10e3583ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:47.653152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81290281-5c4a-457a-9f25-65f10e3583ae None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:47.656877 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854eb182-2a3a-46da-812b-6f594d557706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:47.657512 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854eb182-2a3a-46da-812b-6f594d557706 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:47.657797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854eb182-2a3a-46da-812b-6f594d557706 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:47.658100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854eb182-2a3a-46da-812b-6f594d557706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:47.716799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854eb182-2a3a-46da-812b-6f594d557706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:47.718245 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854eb182-2a3a-46da-812b-6f594d557706 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:47.722768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:47.727458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1260/2542] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:49.979792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c33987-b0bf-44c1-86e9-c3037408d850 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:49.981753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e5c33987-b0bf-44c1-86e9-c3037408d850 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:50.003638 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:50.065363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c33987-b0bf-44c1-86e9-c3037408d850 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:50.073219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df78d3-e2cb-4a62-83ad-8c49a7d0a780 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:50.073892 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df78d3-e2cb-4a62-83ad-8c49a7d0a780 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:50.074337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df78d3-e2cb-4a62-83ad-8c49a7d0a780 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:50.074833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df78d3-e2cb-4a62-83ad-8c49a7d0a780 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:50.076960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f7df78d3-e2cb-4a62-83ad-8c49a7d0a780 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:50.091271 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df78d3-e2cb-4a62-83ad-8c49a7d0a780 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:50.092917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df78d3-e2cb-4a62-83ad-8c49a7d0a780 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:50.105131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1283/2543] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:51.715433 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854eb182-2a3a-46da-812b-6f594d557706 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:51.796961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854eb182-2a3a-46da-812b-6f594d557706 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:51.796961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:51.796961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:51.796961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:51.797422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] PATH_INFO: `/v3/users/903d2a5c0d094e469e9433451bd53d23` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:51.797422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=903d2a5c0d094e469e9433451bd53d23)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:51.797422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:51.975862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '903d2a5c0d094e469e9433451bd53d23'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:51.983864 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '903d2a5c0d094e469e9433451bd53d23'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:51.995813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '903d2a5c0d094e469e9433451bd53d23'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:52.076187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '903d2a5c0d094e469e9433451bd53d23'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:52.076797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '903d2a5c0d094e469e9433451bd53d23'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:52.083131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1261/2544] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:11:51 2022] DELETE /identity/v3/users/903d2a5c0d094e469e9433451bd53d23 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:11:52.087612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61406a4-48a2-4e25-9e78-c3c1c7807950 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:52.088069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61406a4-48a2-4e25-9e78-c3c1c7807950 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:52.088165 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61406a4-48a2-4e25-9e78-c3c1c7807950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:52.106206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c61406a4-48a2-4e25-9e78-c3c1c7807950 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:52.107472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c61406a4-48a2-4e25-9e78-c3c1c7807950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:52.178425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1284/2545] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:52.188924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:52.190514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:52.256022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb73415-a2aa-4dd2-ae3e-6962d32d0947 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:52.260370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62066f95-dcbb-41bd-a1f8-ec828901c32a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:52.261008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62066f95-dcbb-41bd-a1f8-ec828901c32a service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:52.261287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62066f95-dcbb-41bd-a1f8-ec828901c32a service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:52.261556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62066f95-dcbb-41bd-a1f8-ec828901c32a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:52.272222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62066f95-dcbb-41bd-a1f8-ec828901c32a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:52.273846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62066f95-dcbb-41bd-a1f8-ec828901c32a service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:52.284879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1262/2546] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:11:52.689953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35714a8d-9e48-409b-8656-741eabf4c2d5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:52.690321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35714a8d-9e48-409b-8656-741eabf4c2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:52.690673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35714a8d-9e48-409b-8656-741eabf4c2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:52.709428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-35714a8d-9e48-409b-8656-741eabf4c2d5 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:52.710771 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-35714a8d-9e48-409b-8656-741eabf4c2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:52.753533 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62066f95-dcbb-41bd-a1f8-ec828901c32a service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:52.755874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-62066f95-dcbb-41bd-a1f8-ec828901c32a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:52.772434 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62066f95-dcbb-41bd-a1f8-ec828901c32a service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:52.776526 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1285/2547] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:11:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:11:52.777228 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:52.778387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:52.778738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:52.779111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] PATH_INFO: `/v3/projects/787f62f60be24ce49921e5017d48d768` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:52.780616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=787f62f60be24ce49921e5017d48d768)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:52.782681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:52.791930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35714a8d-9e48-409b-8656-741eabf4c2d5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:52.794066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-35714a8d-9e48-409b-8656-741eabf4c2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:52.802184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:52.850447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:52.873648 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35714a8d-9e48-409b-8656-741eabf4c2d5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:52.880422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '787f62f60be24ce49921e5017d48d768', 'request_id': 'req-76fd255e-df7b-4760-ac53-d191670b3c5a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:52.889186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '787f62f60be24ce49921e5017d48d768', 'request_id': 'req-76fd255e-df7b-4760-ac53-d191670b3c5a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:52.942900 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5dbf4-3101-4420-9386-740595d58ad4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:52.943557 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5dbf4-3101-4420-9386-740595d58ad4 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:52.947342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5dbf4-3101-4420-9386-740595d58ad4 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:52.947342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5dbf4-3101-4420-9386-740595d58ad4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:52.957155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '787f62f60be24ce49921e5017d48d768', 'request_id': 'req-76fd255e-df7b-4760-ac53-d191670b3c5a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:52.971206 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] The token cache is being invalidate because project 787f62f60be24ce49921e5017d48d768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:11:52.971508 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:52.972647 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:52.974997 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1263/2548] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:11:52 2022] DELETE /identity/v3/projects/787f62f60be24ce49921e5017d48d768 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:11:52.979080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:52.997032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:53.013211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5dbf4-3101-4420-9386-740595d58ad4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:53.015316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5dbf4-3101-4420-9386-740595d58ad4 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:53.030257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1286/2549] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:11:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:11:53.067586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd255e-df7b-4760-ac53-d191670b3c5a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:53.072394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:53.073348 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:53.073666 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:53.073932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] PATH_INFO: `/v3/users/ee64dea625f3445ba1c322edfed33ed3` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:53.075312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee64dea625f3445ba1c322edfed33ed3)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:53.077221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:53.238224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee64dea625f3445ba1c322edfed33ed3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:53.247139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee64dea625f3445ba1c322edfed33ed3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:53.264053 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee64dea625f3445ba1c322edfed33ed3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:53.317072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee64dea625f3445ba1c322edfed33ed3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:53.320782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee64dea625f3445ba1c322edfed33ed3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:53.328672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1264/2550] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:11:52 2022] DELETE /identity/v3/users/ee64dea625f3445ba1c322edfed33ed3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:53.772285 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5dbf4-3101-4420-9386-740595d58ad4 service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:53.796090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5dbf4-3101-4420-9386-740595d58ad4 service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:53.871903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:53.873114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:53.873114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:53.873114 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] PATH_INFO: `/v3/projects/551609a93a4149c8aab15ebfc281f3e1` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:53.874719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=551609a93a4149c8aab15ebfc281f3e1)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:53.876651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:53.984065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:53.996293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551609a93a4149c8aab15ebfc281f3e1', 'request_id': 'req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:54.002798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551609a93a4149c8aab15ebfc281f3e1', 'request_id': 'req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:54.009976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551609a93a4149c8aab15ebfc281f3e1', 'request_id': 'req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:54.096446 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] The token cache is being invalidate because project 551609a93a4149c8aab15ebfc281f3e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:11:54.096749 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:54.099340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:54.101158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1287/2551] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:11:53 2022] DELETE /identity/v3/projects/551609a93a4149c8aab15ebfc281f3e1 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:54.105540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:54.108346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:54.146797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0820d1e-e4b6-4fc2-afb2-dd0c06d0226b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:54.187255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:54.249389 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:54.260328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:54.260328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:54.260328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] PATH_INFO: `/v3/users/b12655a37dab412591ab5e4b86138657` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:54.262688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12655a37dab412591ab5e4b86138657)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:54.264939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:54.451250 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12655a37dab412591ab5e4b86138657'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:54.460363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:54.462301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:54.473936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12655a37dab412591ab5e4b86138657'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:54.496880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12655a37dab412591ab5e4b86138657'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:54.577019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636bd3f-17b0-4a5c-ab10-94897bb5fdb2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:54.584206 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e835b-7e34-4b40-81b9-54cd4b53d1ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:54.585143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e835b-7e34-4b40-81b9-54cd4b53d1ff service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:54.585504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e835b-7e34-4b40-81b9-54cd4b53d1ff service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:54.585793 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e835b-7e34-4b40-81b9-54cd4b53d1ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:54.587815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-360e835b-7e34-4b40-81b9-54cd4b53d1ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:54.591446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12655a37dab412591ab5e4b86138657'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:54.592064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12655a37dab412591ab5e4b86138657'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:54.606650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1265/2552] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:11:54 2022] DELETE /identity/v3/users/b12655a37dab412591ab5e4b86138657 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:54.620227 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e835b-7e34-4b40-81b9-54cd4b53d1ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:54.622140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e835b-7e34-4b40-81b9-54cd4b53d1ff service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:54.637934 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1288/2553] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:11:55.115292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:55.116790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:55.205074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e83f61-cd01-485e-bc2a-97a7825a9c81 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:55.210672 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:55.211800 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:55.212179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:55.212592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] PATH_INFO: `/v3/projects/51c6348829b74684836045fc5c61fee4` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:55.213876 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c6348829b74684836045fc5c61fee4)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:55.216746 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:55.366757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:55.373888 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c6348829b74684836045fc5c61fee4', 'request_id': 'req-f26c10cf-f7b1-42ea-b393-19f49f7b8154'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:55.384214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c6348829b74684836045fc5c61fee4', 'request_id': 'req-f26c10cf-f7b1-42ea-b393-19f49f7b8154'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:55.460887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c6348829b74684836045fc5c61fee4', 'request_id': 'req-f26c10cf-f7b1-42ea-b393-19f49f7b8154'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:55.488726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] The token cache is being invalidate because project 51c6348829b74684836045fc5c61fee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:11:55.488726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:55.489340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f26c10cf-f7b1-42ea-b393-19f49f7b8154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:11:55.504904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1266/2554] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:11:55 2022] DELETE /identity/v3/projects/51c6348829b74684836045fc5c61fee4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:55.505097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9955bcf-fe6d-4869-b44f-d7f9cbbdeb20 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:55.505097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9955bcf-fe6d-4869-b44f-d7f9cbbdeb20 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:55.505097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9955bcf-fe6d-4869-b44f-d7f9cbbdeb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:55.529246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f9955bcf-fe6d-4869-b44f-d7f9cbbdeb20 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:55.530571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f9955bcf-fe6d-4869-b44f-d7f9cbbdeb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:55.615345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1289/2555] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:55.626541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda60459-f346-4459-9bec-c02cb34f382e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:55.626975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda60459-f346-4459-9bec-c02cb34f382e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:55.627395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda60459-f346-4459-9bec-c02cb34f382e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:55.647573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-eda60459-f346-4459-9bec-c02cb34f382e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:55.648879 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-eda60459-f346-4459-9bec-c02cb34f382e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:55.660751 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1267/2556] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:55.664932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9955bcf-fe6d-4869-b44f-d7f9cbbdeb20 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:55.677870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9955bcf-fe6d-4869-b44f-d7f9cbbdeb20 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:55.682652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb0a05-7d33-4aeb-8751-ceddb3a665fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:55.686883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb0a05-7d33-4aeb-8751-ceddb3a665fa admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:55.686883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb0a05-7d33-4aeb-8751-ceddb3a665fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:55.686883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb0a05-7d33-4aeb-8751-ceddb3a665fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:55.686883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb0a05-7d33-4aeb-8751-ceddb3a665fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:11:55.686883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb0a05-7d33-4aeb-8751-ceddb3a665fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:55.687804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb0a05-7d33-4aeb-8751-ceddb3a665fa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:55.706001 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1290/2557] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:11:55.709380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda60459-f346-4459-9bec-c02cb34f382e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:55.732842 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda60459-f346-4459-9bec-c02cb34f382e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:55.738590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dc3d2-52f9-4064-9aa1-96f363c4594e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:55.739247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dc3d2-52f9-4064-9aa1-96f363c4594e admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:55.739529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dc3d2-52f9-4064-9aa1-96f363c4594e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:55.739808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dc3d2-52f9-4064-9aa1-96f363c4594e admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:55.740489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884dc3d2-52f9-4064-9aa1-96f363c4594e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:55.742429 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884dc3d2-52f9-4064-9aa1-96f363c4594e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:55.819628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1268/2558] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:11:55 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:55.821167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb0a05-7d33-4aeb-8751-ceddb3a665fa admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:55.833454 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb0a05-7d33-4aeb-8751-ceddb3a665fa admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:55.892296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:11:55.914077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a96ff8-0561-4539-9f31-8981945cce70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:55.914077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a96ff8-0561-4539-9f31-8981945cce70 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:55.914077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a96ff8-0561-4539-9f31-8981945cce70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:55.914543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a96ff8-0561-4539-9f31-8981945cce70 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:55.915140 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a96ff8-0561-4539-9f31-8981945cce70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:55.917738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a96ff8-0561-4539-9f31-8981945cce70 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:56.544681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1291/2559] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:11:55 2022] POST /identity/v3/users => generated 291 bytes in 721 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:11:56.548638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dc3d2-52f9-4064-9aa1-96f363c4594e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:56.560153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dc3d2-52f9-4064-9aa1-96f363c4594e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:56.567499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1cd65-e3b6-4482-8093-21ea4caa2b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:56.570962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1cd65-e3b6-4482-8093-21ea4caa2b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:56.570962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1cd65-e3b6-4482-8093-21ea4caa2b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:56.570962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1cd65-e3b6-4482-8093-21ea4caa2b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:56.570962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1cd65-e3b6-4482-8093-21ea4caa2b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:56.571377 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1cd65-e3b6-4482-8093-21ea4caa2b0f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:56.582203 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1269/2560] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:11:56.586534 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a96ff8-0561-4539-9f31-8981945cce70 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:56.599247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a96ff8-0561-4539-9f31-8981945cce70 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:56.611761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c4dad-197a-479d-a1ea-38cad772a213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:56.612510 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c4dad-197a-479d-a1ea-38cad772a213 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:56.612800 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c4dad-197a-479d-a1ea-38cad772a213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:56.613076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c4dad-197a-479d-a1ea-38cad772a213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:56.613984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c4dad-197a-479d-a1ea-38cad772a213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:56.620741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c4dad-197a-479d-a1ea-38cad772a213 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:56.630076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1292/2561] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:11:56.633324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1cd65-e3b6-4482-8093-21ea4caa2b0f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:56.655559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1cd65-e3b6-4482-8093-21ea4caa2b0f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:56.663335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8a91e-ee89-4c09-908e-be60ef300e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:56.664400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8a91e-ee89-4c09-908e-be60ef300e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:56.664779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8a91e-ee89-4c09-908e-be60ef300e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:56.665162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8a91e-ee89-4c09-908e-be60ef300e25 admin admin] PATH_INFO: `/v3/projects/2362e98f5387452893f972175548189a/users/8ff085ab5e6d4d278b72b7b8c528c8d4/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:56.702486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8a91e-ee89-4c09-908e-be60ef300e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2362e98f5387452893f972175548189a, user_id=8ff085ab5e6d4d278b72b7b8c528c8d4, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:56.707975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8a91e-ee89-4c09-908e-be60ef300e25 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:56.787116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1270/2562] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:11:56 2022] PUT /identity/v3/projects/2362e98f5387452893f972175548189a/users/8ff085ab5e6d4d278b72b7b8c528c8d4/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:11:56.793704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906117b-3c6c-4144-b42b-6796d7b8d007 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:56.794683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906117b-3c6c-4144-b42b-6796d7b8d007 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:56.795340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906117b-3c6c-4144-b42b-6796d7b8d007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:56.825001 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e906117b-3c6c-4144-b42b-6796d7b8d007 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:56.826314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e906117b-3c6c-4144-b42b-6796d7b8d007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:56.902450 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1293/2563] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:56.913546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8a91e-ee89-4c09-908e-be60ef300e25 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:56.915448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1fa8a91e-ee89-4c09-908e-be60ef300e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:56.983865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8a91e-ee89-4c09-908e-be60ef300e25 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:56.988240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45371c96-8dcb-44e0-bfca-e5b0eb6abe15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:56.990536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45371c96-8dcb-44e0-bfca-e5b0eb6abe15 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:56.990536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45371c96-8dcb-44e0-bfca-e5b0eb6abe15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:56.990536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45371c96-8dcb-44e0-bfca-e5b0eb6abe15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:57.000268 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45371c96-8dcb-44e0-bfca-e5b0eb6abe15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:57.001616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45371c96-8dcb-44e0-bfca-e5b0eb6abe15 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:57.014831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1271/2564] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:11:58.539961 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd813e-aed8-4f97-9641-87cd791fc93f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:58.540453 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd813e-aed8-4f97-9641-87cd791fc93f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:58.540997 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd813e-aed8-4f97-9641-87cd791fc93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:58.589459 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3abd813e-aed8-4f97-9641-87cd791fc93f None None] MFA Rules not processed for user `8ff085ab5e6d4d278b72b7b8c528c8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:11:58.591542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3abd813e-aed8-4f97-9641-87cd791fc93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:58.688438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1294/2565] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:11:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:11:58.700590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45371c96-8dcb-44e0-bfca-e5b0eb6abe15 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:11:58.702081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-45371c96-8dcb-44e0-bfca-e5b0eb6abe15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:11:58.780341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45371c96-8dcb-44e0-bfca-e5b0eb6abe15 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:11:58.786227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda8f53-aa4e-4c0f-88df-31f38f05bb22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:11:58.787422 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda8f53-aa4e-4c0f-88df-31f38f05bb22 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:11:58.787668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda8f53-aa4e-4c0f-88df-31f38f05bb22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:11:58.787949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda8f53-aa4e-4c0f-88df-31f38f05bb22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:11:58.800862 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda8f53-aa4e-4c0f-88df-31f38f05bb22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:11:58.801805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda8f53-aa4e-4c0f-88df-31f38f05bb22 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:11:58.814119 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1272/2566] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:11:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:12:00.620903 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f10009-ae6a-430b-90ed-172ee6564a3c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:00.621456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f10009-ae6a-430b-90ed-172ee6564a3c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:00.621456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f10009-ae6a-430b-90ed-172ee6564a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:00.640678 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-24f10009-ae6a-430b-90ed-172ee6564a3c None None] MFA Rules not processed for user `8ff085ab5e6d4d278b72b7b8c528c8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:12:00.641910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-24f10009-ae6a-430b-90ed-172ee6564a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:00.649607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1295/2567] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:12:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:12:00.985325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda8f53-aa4e-4c0f-88df-31f38f05bb22 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:00.995528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda8f53-aa4e-4c0f-88df-31f38f05bb22 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:00.999615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27785ce8-184f-4a96-8455-813a4f2191d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:01.000470 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27785ce8-184f-4a96-8455-813a4f2191d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:01.000907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27785ce8-184f-4a96-8455-813a4f2191d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:01.001323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27785ce8-184f-4a96-8455-813a4f2191d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:01.003490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-27785ce8-184f-4a96-8455-813a4f2191d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:01.017177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27785ce8-184f-4a96-8455-813a4f2191d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:01.019505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27785ce8-184f-4a96-8455-813a4f2191d1 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:01.034886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1273/2568] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:12:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:12:04.322859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f10009-ae6a-430b-90ed-172ee6564a3c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:04.352775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f10009-ae6a-430b-90ed-172ee6564a3c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:04.355090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168e64f-0ee5-4aa4-b272-ba44a1cdcc66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:04.355763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168e64f-0ee5-4aa4-b272-ba44a1cdcc66 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:04.356029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168e64f-0ee5-4aa4-b272-ba44a1cdcc66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:04.356309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168e64f-0ee5-4aa4-b272-ba44a1cdcc66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:04.358850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c168e64f-0ee5-4aa4-b272-ba44a1cdcc66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:04.401179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:12:04.436159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168e64f-0ee5-4aa4-b272-ba44a1cdcc66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:04.437773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168e64f-0ee5-4aa4-b272-ba44a1cdcc66 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:04.451693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1296/2569] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:12:04 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:12:06.846281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27785ce8-184f-4a96-8455-813a4f2191d1 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:06.850066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-27785ce8-184f-4a96-8455-813a4f2191d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:06.852961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:12:06.923184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27785ce8-184f-4a96-8455-813a4f2191d1 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:06.928568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfbb06-870f-49a3-a8e2-0c43aa63ec9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:06.929205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfbb06-870f-49a3-a8e2-0c43aa63ec9c service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:06.929487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfbb06-870f-49a3-a8e2-0c43aa63ec9c service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:06.929755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfbb06-870f-49a3-a8e2-0c43aa63ec9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:06.931541 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-80dfbb06-870f-49a3-a8e2-0c43aa63ec9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:06.944383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfbb06-870f-49a3-a8e2-0c43aa63ec9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:06.946154 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfbb06-870f-49a3-a8e2-0c43aa63ec9c service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:06.958500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1274/2570] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:12:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:12:12.591256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeec84-3354-452a-8ff9-47bac27f1d58 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:12.591256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeec84-3354-452a-8ff9-47bac27f1d58 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:12.591256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeec84-3354-452a-8ff9-47bac27f1d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:12.609767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8bfeec84-3354-452a-8ff9-47bac27f1d58 None None] MFA Rules not processed for user `6f101dab08684c2483f67a9357efeae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:12:12.610747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8bfeec84-3354-452a-8ff9-47bac27f1d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:12.668153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1297/2571] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:12:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:12:14.239333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ad5e6-f3fa-4e7d-857d-ca4969e61972 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:14.239726 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ad5e6-f3fa-4e7d-857d-ca4969e61972 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:14.239883 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ad5e6-f3fa-4e7d-857d-ca4969e61972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:14.261274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ca1ad5e6-f3fa-4e7d-857d-ca4969e61972 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:12:14.266384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ca1ad5e6-f3fa-4e7d-857d-ca4969e61972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:14.338627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1275/2572] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:12:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:12:14.355010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeec84-3354-452a-8ff9-47bac27f1d58 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:14.357025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8bfeec84-3354-452a-8ff9-47bac27f1d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:14.441316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeec84-3354-452a-8ff9-47bac27f1d58 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:14.449034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db793e-17ae-4f02-a76f-402a48f07ced service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:14.451343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db793e-17ae-4f02-a76f-402a48f07ced service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:14.451343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db793e-17ae-4f02-a76f-402a48f07ced service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:14.451343 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db793e-17ae-4f02-a76f-402a48f07ced service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:14.463089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db793e-17ae-4f02-a76f-402a48f07ced service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:14.465153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db793e-17ae-4f02-a76f-402a48f07ced service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:14.476530 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1298/2573] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:12:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:12:24.330338 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=58 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:12:24.471919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=70 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:12:32.657727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c57e6-f5f4-4c4b-8fce-3409719bf7ca None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:32.658261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c57e6-f5f4-4c4b-8fce-3409719bf7ca None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:32.658464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c57e6-f5f4-4c4b-8fce-3409719bf7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:32.698921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-813c57e6-f5f4-4c4b-8fce-3409719bf7ca None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:12:32.700129 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-813c57e6-f5f4-4c4b-8fce-3409719bf7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:32.728907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1276/2574] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 13 09:12:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:12:32.741958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db793e-17ae-4f02-a76f-402a48f07ced service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:32.788490 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db793e-17ae-4f02-a76f-402a48f07ced service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:32.793605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab5d23-a17d-433e-9130-e98eb962ff70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:32.794283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab5d23-a17d-433e-9130-e98eb962ff70 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:32.794564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab5d23-a17d-433e-9130-e98eb962ff70 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:32.794837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab5d23-a17d-433e-9130-e98eb962ff70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:32.804318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab5d23-a17d-433e-9130-e98eb962ff70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:32.806957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab5d23-a17d-433e-9130-e98eb962ff70 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:32.816664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1299/2575] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:12:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:12:42.726779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=20 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:12:42.813172 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=20 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:12:53.809394 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c57e6-f5f4-4c4b-8fce-3409719bf7ca None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:53.810866 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-813c57e6-f5f4-4c4b-8fce-3409719bf7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:53.844951 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c57e6-f5f4-4c4b-8fce-3409719bf7ca None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:53.848701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b18bb6-a011-444a-a0ae-7375782605d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:53.849339 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b18bb6-a011-444a-a0ae-7375782605d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:53.849615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b18bb6-a011-444a-a0ae-7375782605d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:53.849885 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b18bb6-a011-444a-a0ae-7375782605d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:53.851678 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-87b18bb6-a011-444a-a0ae-7375782605d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:53.887553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b18bb6-a011-444a-a0ae-7375782605d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:53.888892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b18bb6-a011-444a-a0ae-7375782605d6 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:53.908283 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1277/2576] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:12:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:12:57.413321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab5d23-a17d-433e-9130-e98eb962ff70 service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:57.413321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-88ab5d23-a17d-433e-9130-e98eb962ff70 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:57.483238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab5d23-a17d-433e-9130-e98eb962ff70 service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:57.488386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a887dfc-bb7e-4ef2-8aa5-3e7e99716ead service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:57.489699 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a887dfc-bb7e-4ef2-8aa5-3e7e99716ead service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:57.489984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a887dfc-bb7e-4ef2-8aa5-3e7e99716ead service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:57.490292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a887dfc-bb7e-4ef2-8aa5-3e7e99716ead service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:57.492211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4a887dfc-bb7e-4ef2-8aa5-3e7e99716ead service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:57.504083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a887dfc-bb7e-4ef2-8aa5-3e7e99716ead service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:57.506598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a887dfc-bb7e-4ef2-8aa5-3e7e99716ead service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:57.517364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1300/2577] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:12:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:12:57.529941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b18bb6-a011-444a-a0ae-7375782605d6 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:57.532054 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-87b18bb6-a011-444a-a0ae-7375782605d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:57.550485 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b18bb6-a011-444a-a0ae-7375782605d6 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:57.556181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:57.557343 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:57.557625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:57.557894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] PATH_INFO: `/v3/users/35c7c83855084c188ab8a6dacb57a24b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:57.559559 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c7c83855084c188ab8a6dacb57a24b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:57.561685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:57.690635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c7c83855084c188ab8a6dacb57a24b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:12:57.698418 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c7c83855084c188ab8a6dacb57a24b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:12:57.711009 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c7c83855084c188ab8a6dacb57a24b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:12:57.761893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c7c83855084c188ab8a6dacb57a24b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:12:57.762557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-08001339-fb9b-4deb-8e82-ef16f476578b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c7c83855084c188ab8a6dacb57a24b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:12:57.768590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1278/2578] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:12:57 2022] DELETE /identity/v3/users/35c7c83855084c188ab8a6dacb57a24b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:12:58.012563 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a887dfc-bb7e-4ef2-8aa5-3e7e99716ead service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:58.025527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a887dfc-bb7e-4ef2-8aa5-3e7e99716ead service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:58.085678 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:58.086654 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:58.086937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:58.087231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] PATH_INFO: `/v3/projects/eb9e230019f042dbbc680fc3b1340d41` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:58.088386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9e230019f042dbbc680fc3b1340d41)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:58.090337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:58.172366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9e230019f042dbbc680fc3b1340d41', 'request_id': 'req-06bb1072-602a-4f76-8347-a11c09a0408d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:12:58.178358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9e230019f042dbbc680fc3b1340d41', 'request_id': 'req-06bb1072-602a-4f76-8347-a11c09a0408d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:12:58.184464 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9e230019f042dbbc680fc3b1340d41', 'request_id': 'req-06bb1072-602a-4f76-8347-a11c09a0408d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:12:58.229215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] The token cache is being invalidate because project eb9e230019f042dbbc680fc3b1340d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:12:58.229517 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:12:58.230151 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-06bb1072-602a-4f76-8347-a11c09a0408d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:12:58.232154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1301/2579] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:12:58 2022] DELETE /identity/v3/projects/eb9e230019f042dbbc680fc3b1340d41 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:12:58.236569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca5ac8-f3c1-45a9-a3cf-e89a0c65f8e9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:58.236861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca5ac8-f3c1-45a9-a3cf-e89a0c65f8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:58.237143 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca5ac8-f3c1-45a9-a3cf-e89a0c65f8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:58.296357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7bca5ac8-f3c1-45a9-a3cf-e89a0c65f8e9 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:12:58.297580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7bca5ac8-f3c1-45a9-a3cf-e89a0c65f8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:58.333870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:12:58.371685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1279/2580] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:12:58.375618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3c57-0ad7-4f31-9cff-c194574bd786 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:58.375618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3c57-0ad7-4f31-9cff-c194574bd786 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:58.375618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3c57-0ad7-4f31-9cff-c194574bd786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:58.393922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-08ef3c57-0ad7-4f31-9cff-c194574bd786 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:12:58.395394 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-08ef3c57-0ad7-4f31-9cff-c194574bd786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:58.403545 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1302/2581] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:12:58.407915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca5ac8-f3c1-45a9-a3cf-e89a0c65f8e9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:58.418148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca5ac8-f3c1-45a9-a3cf-e89a0c65f8e9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:58.422758 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b784885-9c07-4ddf-a2f7-fa2c6ed74dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:58.423411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b784885-9c07-4ddf-a2f7-fa2c6ed74dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:58.423695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b784885-9c07-4ddf-a2f7-fa2c6ed74dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:58.423973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b784885-9c07-4ddf-a2f7-fa2c6ed74dfd admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:58.424696 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b784885-9c07-4ddf-a2f7-fa2c6ed74dfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:12:58.425005 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b784885-9c07-4ddf-a2f7-fa2c6ed74dfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:58.427251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b784885-9c07-4ddf-a2f7-fa2c6ed74dfd admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:58.438790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1280/2582] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:12:58.443695 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef3c57-0ad7-4f31-9cff-c194574bd786 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:58.452891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef3c57-0ad7-4f31-9cff-c194574bd786 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:58.457483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd56fad-522e-47ed-ae38-d8731ffe670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:58.458352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd56fad-522e-47ed-ae38-d8731ffe670d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:58.458632 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd56fad-522e-47ed-ae38-d8731ffe670d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:58.458902 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd56fad-522e-47ed-ae38-d8731ffe670d admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:58.459590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd56fad-522e-47ed-ae38-d8731ffe670d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:58.461466 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd56fad-522e-47ed-ae38-d8731ffe670d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:58.470929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:12:58.563754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1303/2583] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:12:58 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:12:58.568022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b784885-9c07-4ddf-a2f7-fa2c6ed74dfd admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:58.578450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b784885-9c07-4ddf-a2f7-fa2c6ed74dfd admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:58.635388 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942c597-63d0-443f-bfda-f742b55c26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:58.636046 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942c597-63d0-443f-bfda-f742b55c26c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:58.636326 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942c597-63d0-443f-bfda-f742b55c26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:58.636595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942c597-63d0-443f-bfda-f742b55c26c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:58.637267 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942c597-63d0-443f-bfda-f742b55c26c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:58.639194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942c597-63d0-443f-bfda-f742b55c26c2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:58.715114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1281/2584] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:12:58 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:12:58.716222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd56fad-522e-47ed-ae38-d8731ffe670d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:58.729836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd56fad-522e-47ed-ae38-d8731ffe670d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:58.736610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d8da4-c65e-47fe-a144-cb7eded74a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:58.737032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d8da4-c65e-47fe-a144-cb7eded74a7f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:58.737312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d8da4-c65e-47fe-a144-cb7eded74a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:58.737585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d8da4-c65e-47fe-a144-cb7eded74a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:58.738346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d8da4-c65e-47fe-a144-cb7eded74a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:58.740521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d8da4-c65e-47fe-a144-cb7eded74a7f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:58.750792 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1304/2585] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:12:58.755551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942c597-63d0-443f-bfda-f742b55c26c2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:58.765705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942c597-63d0-443f-bfda-f742b55c26c2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:58.771011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a37889-288e-4697-99db-e400d30d0a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:58.771818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a37889-288e-4697-99db-e400d30d0a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:58.772165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a37889-288e-4697-99db-e400d30d0a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:58.772556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a37889-288e-4697-99db-e400d30d0a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:58.773281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a37889-288e-4697-99db-e400d30d0a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:58.774897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a37889-288e-4697-99db-e400d30d0a91 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:58.785201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1282/2586] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:12:58.789795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d8da4-c65e-47fe-a144-cb7eded74a7f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:58.800270 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d8da4-c65e-47fe-a144-cb7eded74a7f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:58.805993 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c553cf9-3055-4d3c-b11f-bbffa75c9ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:58.806886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c553cf9-3055-4d3c-b11f-bbffa75c9ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:58.807164 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c553cf9-3055-4d3c-b11f-bbffa75c9ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:58.807436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c553cf9-3055-4d3c-b11f-bbffa75c9ff9 admin admin] PATH_INFO: `/v3/projects/0486a8d3794f4b50a96d3477e14e344f/users/47d7d837aafd4db7bf46346db8b399a6/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:58.822914 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c553cf9-3055-4d3c-b11f-bbffa75c9ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0486a8d3794f4b50a96d3477e14e344f, user_id=47d7d837aafd4db7bf46346db8b399a6, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:58.825531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c553cf9-3055-4d3c-b11f-bbffa75c9ff9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:58.911034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1305/2587] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:12:58 2022] PUT /identity/v3/projects/0486a8d3794f4b50a96d3477e14e344f/users/47d7d837aafd4db7bf46346db8b399a6/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:12:58.916223 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2fb67-3b44-410f-9f65-bce4fbff1a19 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:58.916507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2fb67-3b44-410f-9f65-bce4fbff1a19 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:58.916777 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2fb67-3b44-410f-9f65-bce4fbff1a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:58.935022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d3d2fb67-3b44-410f-9f65-bce4fbff1a19 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:12:58.936264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d3d2fb67-3b44-410f-9f65-bce4fbff1a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:58.997939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1283/2588] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:12:59.008571 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c553cf9-3055-4d3c-b11f-bbffa75c9ff9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:12:59.010362 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0c553cf9-3055-4d3c-b11f-bbffa75c9ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:12:59.076312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c553cf9-3055-4d3c-b11f-bbffa75c9ff9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:12:59.079083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ec559-78ef-4540-b931-dfd4476fa9db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:12:59.079727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ec559-78ef-4540-b931-dfd4476fa9db service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:12:59.080010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ec559-78ef-4540-b931-dfd4476fa9db service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:12:59.080333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ec559-78ef-4540-b931-dfd4476fa9db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:12:59.091159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ec559-78ef-4540-b931-dfd4476fa9db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:12:59.091707 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ec559-78ef-4540-b931-dfd4476fa9db service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:12:59.101256 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1306/2589] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:12:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:13:04.196393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607651e-325b-49f6-afb4-f05756bd49b1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:04.196916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607651e-325b-49f6-afb4-f05756bd49b1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:04.197166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607651e-325b-49f6-afb4-f05756bd49b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:04.226668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d607651e-325b-49f6-afb4-f05756bd49b1 None None] MFA Rules not processed for user `47d7d837aafd4db7bf46346db8b399a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:13:04.227921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d607651e-325b-49f6-afb4-f05756bd49b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:04.265536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:13:04.309321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1284/2590] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:13:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:13:04.327998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ec559-78ef-4540-b931-dfd4476fa9db service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:04.329504 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-281ec559-78ef-4540-b931-dfd4476fa9db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:04.422942 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ec559-78ef-4540-b931-dfd4476fa9db service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:04.429358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396a1b2-466e-48c9-9c83-c7e3766eb8a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:04.430020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396a1b2-466e-48c9-9c83-c7e3766eb8a8 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:04.430346 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396a1b2-466e-48c9-9c83-c7e3766eb8a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:04.430624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396a1b2-466e-48c9-9c83-c7e3766eb8a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:04.439569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396a1b2-466e-48c9-9c83-c7e3766eb8a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:04.441194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396a1b2-466e-48c9-9c83-c7e3766eb8a8 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:04.452909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1307/2591] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:13:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:13:07.143108 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335abd81-3368-4316-b14f-4d8877ac806f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:07.143574 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335abd81-3368-4316-b14f-4d8877ac806f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:07.143684 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335abd81-3368-4316-b14f-4d8877ac806f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:07.162367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-335abd81-3368-4316-b14f-4d8877ac806f None None] MFA Rules not processed for user `47d7d837aafd4db7bf46346db8b399a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:13:07.163587 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-335abd81-3368-4316-b14f-4d8877ac806f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:07.170848 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1285/2592] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:13:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:13:14.446910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=82,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:13:17.156768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=14 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:13:27.038551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46e0dc-c5a2-4ac6-8f55-c4c7d5e58047 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:27.039167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46e0dc-c5a2-4ac6-8f55-c4c7d5e58047 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:27.039167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46e0dc-c5a2-4ac6-8f55-c4c7d5e58047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:27.056877 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4f46e0dc-c5a2-4ac6-8f55-c4c7d5e58047 None None] MFA Rules not processed for user `8ff085ab5e6d4d278b72b7b8c528c8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:13:27.058115 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4f46e0dc-c5a2-4ac6-8f55-c4c7d5e58047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:27.116018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1308/2593] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:13:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:13:32.290227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335abd81-3368-4316-b14f-4d8877ac806f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:32.300099 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335abd81-3368-4316-b14f-4d8877ac806f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:32.303618 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb468a7f-a786-4c5d-976c-4aefb3d72f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:32.304258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb468a7f-a786-4c5d-976c-4aefb3d72f7b service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:32.304535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb468a7f-a786-4c5d-976c-4aefb3d72f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:32.304806 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb468a7f-a786-4c5d-976c-4aefb3d72f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:32.306790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cb468a7f-a786-4c5d-976c-4aefb3d72f7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:32.318929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb468a7f-a786-4c5d-976c-4aefb3d72f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:32.320371 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb468a7f-a786-4c5d-976c-4aefb3d72f7b service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:32.331089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1286/2594] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:13:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:13:36.037232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46e0dc-c5a2-4ac6-8f55-c4c7d5e58047 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:36.038784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4f46e0dc-c5a2-4ac6-8f55-c4c7d5e58047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:36.053493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46e0dc-c5a2-4ac6-8f55-c4c7d5e58047 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:36.058924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:36.063077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:36.063077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:36.063077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] PATH_INFO: `/v3/users/1438ef91441f4493844a915ce12fcfa4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:36.063077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1438ef91441f4493844a915ce12fcfa4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:36.063498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:36.232927 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1438ef91441f4493844a915ce12fcfa4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:13:36.239722 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1438ef91441f4493844a915ce12fcfa4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:13:36.308826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1438ef91441f4493844a915ce12fcfa4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:13:36.309675 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1438ef91441f4493844a915ce12fcfa4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:13:36.310338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c8384fae-2541-4c50-a396-f4cccda05382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1438ef91441f4493844a915ce12fcfa4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:13:36.335174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1309/2595] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:13:36 2022] DELETE /identity/v3/users/1438ef91441f4493844a915ce12fcfa4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:13:36.597581 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb468a7f-a786-4c5d-976c-4aefb3d72f7b service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:36.612564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb468a7f-a786-4c5d-976c-4aefb3d72f7b service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:36.671304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:36.672163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:36.672443 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:36.672716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] PATH_INFO: `/v3/projects/09b63eccabbc498cb96dd4bbcef0969a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:36.673900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b63eccabbc498cb96dd4bbcef0969a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:36.675878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:36.778511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b63eccabbc498cb96dd4bbcef0969a', 'request_id': 'req-34e2987c-ca0c-4b59-b093-93e572f7e123'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:13:36.784163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b63eccabbc498cb96dd4bbcef0969a', 'request_id': 'req-34e2987c-ca0c-4b59-b093-93e572f7e123'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:13:36.858734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b63eccabbc498cb96dd4bbcef0969a', 'request_id': 'req-34e2987c-ca0c-4b59-b093-93e572f7e123'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:13:36.870700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] The token cache is being invalidate because project 09b63eccabbc498cb96dd4bbcef0969a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:13:36.870985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:13:36.871612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34e2987c-ca0c-4b59-b093-93e572f7e123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:13:36.873153 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1287/2596] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:13:36 2022] DELETE /identity/v3/projects/09b63eccabbc498cb96dd4bbcef0969a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:13:36.878297 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da33c4-d23b-49b6-9781-9a2e2de550b0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:36.878588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da33c4-d23b-49b6-9781-9a2e2de550b0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:36.878865 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da33c4-d23b-49b6-9781-9a2e2de550b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:36.897555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e0da33c4-d23b-49b6-9781-9a2e2de550b0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:13:36.898813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e0da33c4-d23b-49b6-9781-9a2e2de550b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:36.958671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1310/2597] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:13:36.967101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b60a0-f1b7-454c-9a6f-cbe912045902 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:36.967387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b60a0-f1b7-454c-9a6f-cbe912045902 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:36.967665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b60a0-f1b7-454c-9a6f-cbe912045902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:36.985742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1a7b60a0-f1b7-454c-9a6f-cbe912045902 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:13:36.986996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1a7b60a0-f1b7-454c-9a6f-cbe912045902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:36.995817 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1288/2598] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:13:36.999814 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da33c4-d23b-49b6-9781-9a2e2de550b0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:37.010475 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da33c4-d23b-49b6-9781-9a2e2de550b0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:37.015043 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34882701-2c94-4d3e-b988-d04d3d5ea893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:37.015690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882701-2c94-4d3e-b988-d04d3d5ea893 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:37.015972 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882701-2c94-4d3e-b988-d04d3d5ea893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:37.016246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882701-2c94-4d3e-b988-d04d3d5ea893 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:37.016969 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34882701-2c94-4d3e-b988-d04d3d5ea893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:13:37.017275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34882701-2c94-4d3e-b988-d04d3d5ea893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:37.018913 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34882701-2c94-4d3e-b988-d04d3d5ea893 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:37.031131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1311/2599] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:13:37.035358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b60a0-f1b7-454c-9a6f-cbe912045902 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:37.044761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b60a0-f1b7-454c-9a6f-cbe912045902 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:37.049493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9def7-e9c6-445b-87eb-5ff8f570d7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:37.050172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9def7-e9c6-445b-87eb-5ff8f570d7cb admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:37.050454 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9def7-e9c6-445b-87eb-5ff8f570d7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:37.050727 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9def7-e9c6-445b-87eb-5ff8f570d7cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:37.051416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9def7-e9c6-445b-87eb-5ff8f570d7cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:37.053289 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9def7-e9c6-445b-87eb-5ff8f570d7cb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:37.151859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1289/2600] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:13:37 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:13:37.155979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34882701-2c94-4d3e-b988-d04d3d5ea893 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:37.162395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:13:37.166803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34882701-2c94-4d3e-b988-d04d3d5ea893 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:37.228642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8a206-29fc-4c27-a215-de4273313d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:37.234022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8a206-29fc-4c27-a215-de4273313d9f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:37.234022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8a206-29fc-4c27-a215-de4273313d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:37.234022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8a206-29fc-4c27-a215-de4273313d9f admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:37.234022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8a206-29fc-4c27-a215-de4273313d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:37.234022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8a206-29fc-4c27-a215-de4273313d9f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:37.350789 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1312/2601] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:13:37 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:13:37.355323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9def7-e9c6-445b-87eb-5ff8f570d7cb admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:37.365411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9def7-e9c6-445b-87eb-5ff8f570d7cb admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:37.370749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6efe5-d48b-455e-8310-3517c75f8eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:37.371437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6efe5-d48b-455e-8310-3517c75f8eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:37.371720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6efe5-d48b-455e-8310-3517c75f8eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:37.372035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6efe5-d48b-455e-8310-3517c75f8eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:37.372700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6efe5-d48b-455e-8310-3517c75f8eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:37.374329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6efe5-d48b-455e-8310-3517c75f8eb8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:37.385202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1290/2602] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:13:37.389228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8a206-29fc-4c27-a215-de4273313d9f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:37.399369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8a206-29fc-4c27-a215-de4273313d9f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:37.404129 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c01a8-14a8-487f-87a9-884e7a4ab944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:37.404789 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c01a8-14a8-487f-87a9-884e7a4ab944 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:37.406743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c01a8-14a8-487f-87a9-884e7a4ab944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:37.406743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c01a8-14a8-487f-87a9-884e7a4ab944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:37.406743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c01a8-14a8-487f-87a9-884e7a4ab944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:37.407805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c01a8-14a8-487f-87a9-884e7a4ab944 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:37.419795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1313/2603] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:13:37.423944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6efe5-d48b-455e-8310-3517c75f8eb8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:37.435782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6efe5-d48b-455e-8310-3517c75f8eb8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:37.440432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f8153-1777-48bd-8c94-cb2a9aa93f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:37.441301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f8153-1777-48bd-8c94-cb2a9aa93f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:37.441582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f8153-1777-48bd-8c94-cb2a9aa93f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:37.441856 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f8153-1777-48bd-8c94-cb2a9aa93f4e admin admin] PATH_INFO: `/v3/projects/613ff427420a4e3faf3b73b72cd92668/users/9eab86d45e8b4fefb5c190ec98021cd6/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:37.457954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f8153-1777-48bd-8c94-cb2a9aa93f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=613ff427420a4e3faf3b73b72cd92668, user_id=9eab86d45e8b4fefb5c190ec98021cd6, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:37.460687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f8153-1777-48bd-8c94-cb2a9aa93f4e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:37.558916 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1291/2604] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:13:37 2022] PUT /identity/v3/projects/613ff427420a4e3faf3b73b72cd92668/users/9eab86d45e8b4fefb5c190ec98021cd6/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:13:37.566640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b57ad1-172d-4a04-a2a4-5aaee8eb1411 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:37.566930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b57ad1-172d-4a04-a2a4-5aaee8eb1411 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:37.567231 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b57ad1-172d-4a04-a2a4-5aaee8eb1411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:37.594435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-45b57ad1-172d-4a04-a2a4-5aaee8eb1411 None None] MFA Rules not processed for user `9eab86d45e8b4fefb5c190ec98021cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:13:37.595640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-45b57ad1-172d-4a04-a2a4-5aaee8eb1411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:37.660567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1314/2605] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:13:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:13:37.665432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f8153-1777-48bd-8c94-cb2a9aa93f4e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:37.678760 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:13:37.726845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f8153-1777-48bd-8c94-cb2a9aa93f4e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:37.732193 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91f032-12e5-49d5-b951-1f66c36b2c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:37.732878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91f032-12e5-49d5-b951-1f66c36b2c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:37.733303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91f032-12e5-49d5-b951-1f66c36b2c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:37.733925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91f032-12e5-49d5-b951-1f66c36b2c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:37.734689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91f032-12e5-49d5-b951-1f66c36b2c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:37.736612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91f032-12e5-49d5-b951-1f66c36b2c66 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:37.825513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1292/2606] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:13:37 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:13:37.829576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b57ad1-172d-4a04-a2a4-5aaee8eb1411 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:37.840652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b57ad1-172d-4a04-a2a4-5aaee8eb1411 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:37.896370 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b216d2d-b654-427a-85ea-c77ad0c55846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:37.897035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b216d2d-b654-427a-85ea-c77ad0c55846 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:37.897318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b216d2d-b654-427a-85ea-c77ad0c55846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:37.897594 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b216d2d-b654-427a-85ea-c77ad0c55846 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:37.898318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b216d2d-b654-427a-85ea-c77ad0c55846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:37.900803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b216d2d-b654-427a-85ea-c77ad0c55846 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:37.973212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1315/2607] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:13:37 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:13:37.976950 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91f032-12e5-49d5-b951-1f66c36b2c66 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:37.987966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91f032-12e5-49d5-b951-1f66c36b2c66 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:37.992553 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e81f4-91cd-4862-b9b8-411f3f939b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:37.993224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e81f4-91cd-4862-b9b8-411f3f939b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:37.993570 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e81f4-91cd-4862-b9b8-411f3f939b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:37.993845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e81f4-91cd-4862-b9b8-411f3f939b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:37.994598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e81f4-91cd-4862-b9b8-411f3f939b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:37.996182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e81f4-91cd-4862-b9b8-411f3f939b1a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:38.006745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1293/2608] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:13:38.010965 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b216d2d-b654-427a-85ea-c77ad0c55846 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:38.016141 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:13:38.021442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b216d2d-b654-427a-85ea-c77ad0c55846 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:38.026589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8008bd4-a053-4648-9136-54a3f36f8a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:38.027930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8008bd4-a053-4648-9136-54a3f36f8a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:38.027930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8008bd4-a053-4648-9136-54a3f36f8a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:38.028119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8008bd4-a053-4648-9136-54a3f36f8a48 admin admin] PATH_INFO: `/v3/projects/25a20cb812434de69111c5d39ec5e309/users/7c6375da5d1b451b9ae21c08f600e346/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:38.043535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8008bd4-a053-4648-9136-54a3f36f8a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a20cb812434de69111c5d39ec5e309, user_id=7c6375da5d1b451b9ae21c08f600e346, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:38.046306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8008bd4-a053-4648-9136-54a3f36f8a48 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:38.110972 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1316/2609] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:13:38 2022] PUT /identity/v3/projects/25a20cb812434de69111c5d39ec5e309/users/7c6375da5d1b451b9ae21c08f600e346/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:13:38.114948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e81f4-91cd-4862-b9b8-411f3f939b1a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:38.125623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e81f4-91cd-4862-b9b8-411f3f939b1a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:38.181639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d305d-0ec3-418c-9312-6c006a32a87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:38.182321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d305d-0ec3-418c-9312-6c006a32a87c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:38.182601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d305d-0ec3-418c-9312-6c006a32a87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:38.182878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d305d-0ec3-418c-9312-6c006a32a87c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:38.183598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d305d-0ec3-418c-9312-6c006a32a87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:38.185221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d305d-0ec3-418c-9312-6c006a32a87c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:38.194766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1294/2610] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:13:38.199698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8008bd4-a053-4648-9136-54a3f36f8a48 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:38.209964 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8008bd4-a053-4648-9136-54a3f36f8a48 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:38.215341 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6328ad3-a7e1-42bc-8382-1338cca0711a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:38.216208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328ad3-a7e1-42bc-8382-1338cca0711a admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:38.216487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328ad3-a7e1-42bc-8382-1338cca0711a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:38.216756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328ad3-a7e1-42bc-8382-1338cca0711a admin admin] PATH_INFO: `/v3/projects/25a20cb812434de69111c5d39ec5e309/users/7c6375da5d1b451b9ae21c08f600e346/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:38.218707 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6328ad3-a7e1-42bc-8382-1338cca0711a admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a20cb812434de69111c5d39ec5e309, user_id=7c6375da5d1b451b9ae21c08f600e346, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:38.221320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6328ad3-a7e1-42bc-8382-1338cca0711a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:38.231438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1317/2611] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:13:38 2022] PUT /identity/v3/projects/25a20cb812434de69111c5d39ec5e309/users/7c6375da5d1b451b9ae21c08f600e346/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:13:38.242148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904c6f-5a35-4bc1-ac03-f8b78c62ab18 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:38.242442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904c6f-5a35-4bc1-ac03-f8b78c62ab18 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:38.242722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904c6f-5a35-4bc1-ac03-f8b78c62ab18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:38.269180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b8904c6f-5a35-4bc1-ac03-f8b78c62ab18 None None] MFA Rules not processed for user `7c6375da5d1b451b9ae21c08f600e346`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:13:38.270415 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b8904c6f-5a35-4bc1-ac03-f8b78c62ab18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:38.338720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1295/2612] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:13:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:13:38.350441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6328ad3-a7e1-42bc-8382-1338cca0711a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:38.352171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f6328ad3-a7e1-42bc-8382-1338cca0711a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:13:38.430698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6328ad3-a7e1-42bc-8382-1338cca0711a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:38.435030 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92dd2c-eae4-4109-b5a3-41bf760664e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:38.435667 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92dd2c-eae4-4109-b5a3-41bf760664e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:38.435953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92dd2c-eae4-4109-b5a3-41bf760664e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:38.436230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92dd2c-eae4-4109-b5a3-41bf760664e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:38.445487 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92dd2c-eae4-4109-b5a3-41bf760664e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:38.446963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92dd2c-eae4-4109-b5a3-41bf760664e0 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:38.457118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1318/2613] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:13:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:13:38.702692 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904c6f-5a35-4bc1-ac03-f8b78c62ab18 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:13:38.712357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904c6f-5a35-4bc1-ac03-f8b78c62ab18 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:13:38.716177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2c2d7-8549-4adf-8462-02878d3adc23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:13:38.721733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2c2d7-8549-4adf-8462-02878d3adc23 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:13:38.721733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2c2d7-8549-4adf-8462-02878d3adc23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:13:38.721733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2c2d7-8549-4adf-8462-02878d3adc23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:13:38.723692 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:13:38.775159 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2c2d7-8549-4adf-8462-02878d3adc23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:13:38.776709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2c2d7-8549-4adf-8462-02878d3adc23 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:13:38.785495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1296/2614] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:13:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:13:48.452895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=49,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:13:49.069962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=24 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:18.827808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92dd2c-eae4-4109-b5a3-41bf760664e0 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:18.829292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3d92dd2c-eae4-4109-b5a3-41bf760664e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:18.920495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92dd2c-eae4-4109-b5a3-41bf760664e0 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:18.926382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439177e-4832-4a1f-b98b-6967013c9476 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:18.927036 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439177e-4832-4a1f-b98b-6967013c9476 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:18.927324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439177e-4832-4a1f-b98b-6967013c9476 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:18.927610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439177e-4832-4a1f-b98b-6967013c9476 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:18.929510 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4439177e-4832-4a1f-b98b-6967013c9476 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:18.941673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439177e-4832-4a1f-b98b-6967013c9476 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:18.943310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439177e-4832-4a1f-b98b-6967013c9476 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:18.953505 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1319/2615] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:26.972126 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2c2d7-8549-4adf-8462-02878d3adc23 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:26.976825 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-54b2c2d7-8549-4adf-8462-02878d3adc23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:26.986100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2c2d7-8549-4adf-8462-02878d3adc23 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:26.991633 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec6f7c-a086-430a-be55-4fd8c7012feb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:26.992297 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6f7c-a086-430a-be55-4fd8c7012feb service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:26.992575 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6f7c-a086-430a-be55-4fd8c7012feb service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:26.993059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6f7c-a086-430a-be55-4fd8c7012feb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:26.995310 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4cec6f7c-a086-430a-be55-4fd8c7012feb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:27.061994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec6f7c-a086-430a-be55-4fd8c7012feb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:27.067821 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec6f7c-a086-430a-be55-4fd8c7012feb service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:27.072370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1297/2616] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:28.243007 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0f253-e98b-45ac-9a09-25196258ff51 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:28.243568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0f253-e98b-45ac-9a09-25196258ff51 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:28.243568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0f253-e98b-45ac-9a09-25196258ff51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:28.261611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bcf0f253-e98b-45ac-9a09-25196258ff51 None None] MFA Rules not processed for user `47d7d837aafd4db7bf46346db8b399a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:28.262836 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bcf0f253-e98b-45ac-9a09-25196258ff51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:28.318841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1320/2617] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:33.802150 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec6f7c-a086-430a-be55-4fd8c7012feb service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:33.870076 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec6f7c-a086-430a-be55-4fd8c7012feb service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:33.874740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:33.875614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:33.875893 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:33.876167 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] PATH_INFO: `/v3/users/9eab86d45e8b4fefb5c190ec98021cd6` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:33.877349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eab86d45e8b4fefb5c190ec98021cd6)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:33.879811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:34.080170 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eab86d45e8b4fefb5c190ec98021cd6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:34.090797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eab86d45e8b4fefb5c190ec98021cd6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:34.107146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eab86d45e8b4fefb5c190ec98021cd6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:34.202818 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eab86d45e8b4fefb5c190ec98021cd6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:34.203509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eab86d45e8b4fefb5c190ec98021cd6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:34.212489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1298/2618] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:33 2022] DELETE /identity/v3/users/9eab86d45e8b4fefb5c190ec98021cd6 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:34.217361 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24542d-d29a-414e-b37c-6d061dc72c98 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:34.218257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24542d-d29a-414e-b37c-6d061dc72c98 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:34.218257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24542d-d29a-414e-b37c-6d061dc72c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:34.240932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8c24542d-d29a-414e-b37c-6d061dc72c98 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:34.242570 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8c24542d-d29a-414e-b37c-6d061dc72c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:34.321378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1321/2619] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:34.332789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:34.337935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:34.365774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:34.405779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb402098-9186-411c-a664-43c28c231dc3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:34.409424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040d8ff-3c51-461d-9b06-8a715d51147c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:34.412905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040d8ff-3c51-461d-9b06-8a715d51147c service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:34.413162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040d8ff-3c51-461d-9b06-8a715d51147c service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:34.413437 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040d8ff-3c51-461d-9b06-8a715d51147c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:34.423147 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040d8ff-3c51-461d-9b06-8a715d51147c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:34.424631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040d8ff-3c51-461d-9b06-8a715d51147c service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:34.436674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1299/2620] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:34.946650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24542d-d29a-414e-b37c-6d061dc72c98 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:34.948610 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8c24542d-d29a-414e-b37c-6d061dc72c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:34.981989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24542d-d29a-414e-b37c-6d061dc72c98 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:34.989366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:34.989583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:34.990124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:34.990654 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] PATH_INFO: `/v3/projects/613ff427420a4e3faf3b73b72cd92668` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:34.993679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] RBAC: Authorizing `identity:delete_project(project_id=613ff427420a4e3faf3b73b72cd92668)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:35.005418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:35.008883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:35.129612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613ff427420a4e3faf3b73b72cd92668', 'request_id': 'req-3e75954f-54dc-4843-9b66-1230b11a436d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:35.136385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613ff427420a4e3faf3b73b72cd92668', 'request_id': 'req-3e75954f-54dc-4843-9b66-1230b11a436d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:35.142963 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613ff427420a4e3faf3b73b72cd92668', 'request_id': 'req-3e75954f-54dc-4843-9b66-1230b11a436d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:35.219312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] The token cache is being invalidate because project 613ff427420a4e3faf3b73b72cd92668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:35.219975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:35.220666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3e75954f-54dc-4843-9b66-1230b11a436d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:35.231862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1322/2621] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:34 2022] DELETE /identity/v3/projects/613ff427420a4e3faf3b73b72cd92668 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:35.236020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040d8ff-3c51-461d-9b06-8a715d51147c service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:35.248216 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040d8ff-3c51-461d-9b06-8a715d51147c service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:35.315713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:35.316635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:35.316914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:35.317219 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] PATH_INFO: `/v3/users/7c6375da5d1b451b9ae21c08f600e346` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:35.318546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c6375da5d1b451b9ae21c08f600e346)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:35.320479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:35.528255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c6375da5d1b451b9ae21c08f600e346'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:35.535749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c6375da5d1b451b9ae21c08f600e346'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:35.551186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c6375da5d1b451b9ae21c08f600e346'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:35.697298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c6375da5d1b451b9ae21c08f600e346'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:35.698116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c6375da5d1b451b9ae21c08f600e346'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:35.705220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1300/2622] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:35 2022] DELETE /identity/v3/users/7c6375da5d1b451b9ae21c08f600e346 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:14:35.744128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c771cdd-ef91-4e29-aa84-6743c0ad9473 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:35.744420 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c771cdd-ef91-4e29-aa84-6743c0ad9473 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:35.744703 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c771cdd-ef91-4e29-aa84-6743c0ad9473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:35.763184 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9c771cdd-ef91-4e29-aa84-6743c0ad9473 None None] MFA Rules not processed for user `6f101dab08684c2483f67a9357efeae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:35.764517 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9c771cdd-ef91-4e29-aa84-6743c0ad9473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:35.820932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1323/2623] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:36.256526 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:36.258405 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:36.305826 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:36.328543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d67700-8d03-4f3a-8a7a-fb4dfdfb424d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:36.333646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:36.334540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:36.334824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:36.335094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] PATH_INFO: `/v3/projects/25a20cb812434de69111c5d39ec5e309` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:36.343366 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a20cb812434de69111c5d39ec5e309)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:36.345738 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:36.475639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a20cb812434de69111c5d39ec5e309', 'request_id': 'req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:36.483011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a20cb812434de69111c5d39ec5e309', 'request_id': 'req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:36.572765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a20cb812434de69111c5d39ec5e309', 'request_id': 'req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:36.588322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] The token cache is being invalidate because project 25a20cb812434de69111c5d39ec5e309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:36.588688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:36.589447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1e695d60-ac08-4a27-8dd9-c5cf76def3e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:36.591184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1301/2624] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:36 2022] DELETE /identity/v3/projects/25a20cb812434de69111c5d39ec5e309 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:36.596041 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650efb16-d348-4671-b4ee-01cc6f7c6980 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:36.596840 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650efb16-d348-4671-b4ee-01cc6f7c6980 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:36.597224 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650efb16-d348-4671-b4ee-01cc6f7c6980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:36.630710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-650efb16-d348-4671-b4ee-01cc6f7c6980 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:36.632278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-650efb16-d348-4671-b4ee-01cc6f7c6980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:36.695819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1324/2625] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:36.704165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726d80e-8b81-4338-8258-f3243c30f371 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:36.704455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726d80e-8b81-4338-8258-f3243c30f371 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:36.704741 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726d80e-8b81-4338-8258-f3243c30f371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:36.726144 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8726d80e-8b81-4338-8258-f3243c30f371 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:36.727282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8726d80e-8b81-4338-8258-f3243c30f371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:36.735652 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1302/2626] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:36.741092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650efb16-d348-4671-b4ee-01cc6f7c6980 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:36.750895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650efb16-d348-4671-b4ee-01cc6f7c6980 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:36.755524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe796fa-9fff-4ba5-a229-1ef4dac4f721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:36.756228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe796fa-9fff-4ba5-a229-1ef4dac4f721 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:36.756511 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe796fa-9fff-4ba5-a229-1ef4dac4f721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:36.757041 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe796fa-9fff-4ba5-a229-1ef4dac4f721 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:36.757770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe796fa-9fff-4ba5-a229-1ef4dac4f721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:14:36.758117 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe796fa-9fff-4ba5-a229-1ef4dac4f721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:36.759747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe796fa-9fff-4ba5-a229-1ef4dac4f721 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:36.774786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1325/2627] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:14:36.776177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726d80e-8b81-4338-8258-f3243c30f371 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:36.785977 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726d80e-8b81-4338-8258-f3243c30f371 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:36.790935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fe8c1-50b2-42e2-ba8d-fb02249d7372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:36.791623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fe8c1-50b2-42e2-ba8d-fb02249d7372 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:36.791909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fe8c1-50b2-42e2-ba8d-fb02249d7372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:36.792185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fe8c1-50b2-42e2-ba8d-fb02249d7372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:36.792902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fe8c1-50b2-42e2-ba8d-fb02249d7372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:36.794857 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fe8c1-50b2-42e2-ba8d-fb02249d7372 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:36.896420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1303/2628] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:14:36 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:36.899119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe796fa-9fff-4ba5-a229-1ef4dac4f721 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:36.909281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe796fa-9fff-4ba5-a229-1ef4dac4f721 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:36.939050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:36.966262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b7d30-eaed-4b64-87d5-9e6058c9a983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:36.966966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b7d30-eaed-4b64-87d5-9e6058c9a983 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:36.967284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b7d30-eaed-4b64-87d5-9e6058c9a983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:36.967567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b7d30-eaed-4b64-87d5-9e6058c9a983 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:36.968252 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b7d30-eaed-4b64-87d5-9e6058c9a983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:36.970246 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b7d30-eaed-4b64-87d5-9e6058c9a983 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:37.070317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1326/2629] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:14:36 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:37.071270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fe8c1-50b2-42e2-ba8d-fb02249d7372 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:37.085012 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fe8c1-50b2-42e2-ba8d-fb02249d7372 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:37.094058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3181ec9-cff0-4b90-8b0d-639ccbfc6204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:37.094058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3181ec9-cff0-4b90-8b0d-639ccbfc6204 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.094058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3181ec9-cff0-4b90-8b0d-639ccbfc6204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.094058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3181ec9-cff0-4b90-8b0d-639ccbfc6204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.094677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3181ec9-cff0-4b90-8b0d-639ccbfc6204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:37.094947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3181ec9-cff0-4b90-8b0d-639ccbfc6204 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:37.121056 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1304/2630] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:37.129477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea412af-0c36-4f60-9424-825e44140322 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.129773 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea412af-0c36-4f60-9424-825e44140322 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.130012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea412af-0c36-4f60-9424-825e44140322 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.131880 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1327/2631] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 13 09:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 09:14:37.132705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3181ec9-cff0-4b90-8b0d-639ccbfc6204 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:37.140492 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb04e3b-119e-43d8-a080-f79c13b4b363 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.140786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb04e3b-119e-43d8-a080-f79c13b4b363 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.141077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb04e3b-119e-43d8-a080-f79c13b4b363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.149750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3181ec9-cff0-4b90-8b0d-639ccbfc6204 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:37.154399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85acb3-69b3-4095-8f1e-e3640c6240f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:37.155074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85acb3-69b3-4095-8f1e-e3640c6240f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.155439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85acb3-69b3-4095-8f1e-e3640c6240f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.155637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85acb3-69b3-4095-8f1e-e3640c6240f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.156645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85acb3-69b3-4095-8f1e-e3640c6240f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:37.158625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85acb3-69b3-4095-8f1e-e3640c6240f1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:37.174101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1305/2632] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:37.178927 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85acb3-69b3-4095-8f1e-e3640c6240f1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:37.194711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4cb04e3b-119e-43d8-a080-f79c13b4b363 None None] MFA Rules not processed for user `12824152139841138f457dfe5c9758e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:37.195134 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85acb3-69b3-4095-8f1e-e3640c6240f1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:37.195945 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4cb04e3b-119e-43d8-a080-f79c13b4b363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:37.200072 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c23ef-062d-4730-b037-425e3853d18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:37.200962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c23ef-062d-4730-b037-425e3853d18b admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.201248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c23ef-062d-4730-b037-425e3853d18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.201514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c23ef-062d-4730-b037-425e3853d18b admin admin] PATH_INFO: `/v3/projects/dec2e269c61f4a17bade8696a3693ae6/users/5ccdd3ea4f6c48b1a2d1b8f6130e91d2/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.226998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c23ef-062d-4730-b037-425e3853d18b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec2e269c61f4a17bade8696a3693ae6, user_id=5ccdd3ea4f6c48b1a2d1b8f6130e91d2, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:37.229725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c23ef-062d-4730-b037-425e3853d18b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:37.271731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1328/2633] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 13 09:14:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:14:37.284013 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb04e3b-119e-43d8-a080-f79c13b4b363 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:37.286382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4cb04e3b-119e-43d8-a080-f79c13b4b363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:37.297350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb04e3b-119e-43d8-a080-f79c13b4b363 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:37.302253 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe7619-3bcb-4c11-9be9-25356ca57715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:37.302984 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7619-3bcb-4c11-9be9-25356ca57715 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.303742 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7619-3bcb-4c11-9be9-25356ca57715 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.303742 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7619-3bcb-4c11-9be9-25356ca57715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.312340 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7619-3bcb-4c11-9be9-25356ca57715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:37.314509 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7619-3bcb-4c11-9be9-25356ca57715 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:37.325598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1329/2634] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:14:37.340222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1306/2635] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:37 2022] PUT /identity/v3/projects/dec2e269c61f4a17bade8696a3693ae6/users/5ccdd3ea4f6c48b1a2d1b8f6130e91d2/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:14:37.353123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21320377-5226-4b5c-9f43-d82d15a82972 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.353579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21320377-5226-4b5c-9f43-d82d15a82972 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.353917 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21320377-5226-4b5c-9f43-d82d15a82972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.386018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-21320377-5226-4b5c-9f43-d82d15a82972 None None] MFA Rules not processed for user `5ccdd3ea4f6c48b1a2d1b8f6130e91d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:37.387335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-21320377-5226-4b5c-9f43-d82d15a82972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:37.426598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c23ef-062d-4730-b037-425e3853d18b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:37.467576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1330/2636] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:37.471698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21320377-5226-4b5c-9f43-d82d15a82972 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:37.495722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c23ef-062d-4730-b037-425e3853d18b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:37.499690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b118a6e-55aa-4557-9484-19d91198ecee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:37.500335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b118a6e-55aa-4557-9484-19d91198ecee service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.500615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b118a6e-55aa-4557-9484-19d91198ecee service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.500890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b118a6e-55aa-4557-9484-19d91198ecee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.502857 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2b118a6e-55aa-4557-9484-19d91198ecee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:37.513685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:37.523338 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:37.538455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21320377-5226-4b5c-9f43-d82d15a82972 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:37.543171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ea5e7-6f0f-47df-a728-64760225a635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:37.543832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ea5e7-6f0f-47df-a728-64760225a635 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.544116 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ea5e7-6f0f-47df-a728-64760225a635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.544398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ea5e7-6f0f-47df-a728-64760225a635 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.545090 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ea5e7-6f0f-47df-a728-64760225a635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:37.547080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ea5e7-6f0f-47df-a728-64760225a635 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:37.595427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b118a6e-55aa-4557-9484-19d91198ecee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:37.596943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b118a6e-55aa-4557-9484-19d91198ecee service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:37.617774 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1307/2637] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:37.642101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1331/2638] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:14:37 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:14:37.646649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b118a6e-55aa-4557-9484-19d91198ecee service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:37.658750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b118a6e-55aa-4557-9484-19d91198ecee service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:37.730447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b383d3a-7b6d-4d5f-a5d4-28aafe1e2573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:37.731643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b383d3a-7b6d-4d5f-a5d4-28aafe1e2573 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.732241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b383d3a-7b6d-4d5f-a5d4-28aafe1e2573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.732661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b383d3a-7b6d-4d5f-a5d4-28aafe1e2573 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.733586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b383d3a-7b6d-4d5f-a5d4-28aafe1e2573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:37.736032 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b383d3a-7b6d-4d5f-a5d4-28aafe1e2573 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:37.823530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1308/2639] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:14:37 2022] POST /identity/v3/users => generated 291 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:14:37.825209 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ea5e7-6f0f-47df-a728-64760225a635 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:37.836971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ea5e7-6f0f-47df-a728-64760225a635 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:37.841837 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b829d96-67a4-4df3-a5ce-b25e82e67e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:37.842574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b829d96-67a4-4df3-a5ce-b25e82e67e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.842861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b829d96-67a4-4df3-a5ce-b25e82e67e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.843138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b829d96-67a4-4df3-a5ce-b25e82e67e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.843896 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b829d96-67a4-4df3-a5ce-b25e82e67e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:37.846240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b829d96-67a4-4df3-a5ce-b25e82e67e82 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:37.860368 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1332/2640] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:14:37.861594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b383d3a-7b6d-4d5f-a5d4-28aafe1e2573 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:37.871959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b383d3a-7b6d-4d5f-a5d4-28aafe1e2573 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:37.882117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23d7ae-f2bc-486f-a5a3-15d3332cf3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:37.883075 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23d7ae-f2bc-486f-a5a3-15d3332cf3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:37.883403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23d7ae-f2bc-486f-a5a3-15d3332cf3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:37.883687 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23d7ae-f2bc-486f-a5a3-15d3332cf3a6 admin admin] PATH_INFO: `/v3/projects/7b1a60bae9e84c9d9da7c564af7fb0d0/users/69e42a19cb854f63b5dda9e0cf9c0ec4/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:37.913083 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23d7ae-f2bc-486f-a5a3-15d3332cf3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1a60bae9e84c9d9da7c564af7fb0d0, user_id=69e42a19cb854f63b5dda9e0cf9c0ec4, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:37.916599 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23d7ae-f2bc-486f-a5a3-15d3332cf3a6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:38.005069 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1309/2641] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:37 2022] PUT /identity/v3/projects/7b1a60bae9e84c9d9da7c564af7fb0d0/users/69e42a19cb854f63b5dda9e0cf9c0ec4/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:38.009088 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b829d96-67a4-4df3-a5ce-b25e82e67e82 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:38.020999 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b829d96-67a4-4df3-a5ce-b25e82e67e82 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:38.109119 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30b251-3581-49d3-99a4-161f934a4fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:38.114288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30b251-3581-49d3-99a4-161f934a4fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:38.114288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30b251-3581-49d3-99a4-161f934a4fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:38.114288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30b251-3581-49d3-99a4-161f934a4fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:38.114288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30b251-3581-49d3-99a4-161f934a4fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:38.115364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30b251-3581-49d3-99a4-161f934a4fc8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:38.139048 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1333/2642] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:38.142968 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23d7ae-f2bc-486f-a5a3-15d3332cf3a6 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:38.154290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23d7ae-f2bc-486f-a5a3-15d3332cf3a6 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:38.165657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9569d8-c515-4925-8f89-9061c97294e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:38.171260 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9569d8-c515-4925-8f89-9061c97294e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:38.171655 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9569d8-c515-4925-8f89-9061c97294e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:38.172004 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9569d8-c515-4925-8f89-9061c97294e8 admin admin] PATH_INFO: `/v3/projects/7b1a60bae9e84c9d9da7c564af7fb0d0/users/69e42a19cb854f63b5dda9e0cf9c0ec4/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:38.174850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9569d8-c515-4925-8f89-9061c97294e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1a60bae9e84c9d9da7c564af7fb0d0, user_id=69e42a19cb854f63b5dda9e0cf9c0ec4, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:38.177936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9569d8-c515-4925-8f89-9061c97294e8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:38.195404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1310/2643] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:38 2022] PUT /identity/v3/projects/7b1a60bae9e84c9d9da7c564af7fb0d0/users/69e42a19cb854f63b5dda9e0cf9c0ec4/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:38.205350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9f7a-a7b7-45e8-89c2-86497013ea01 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:38.205730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9f7a-a7b7-45e8-89c2-86497013ea01 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:38.205919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9f7a-a7b7-45e8-89c2-86497013ea01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:38.241097 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-89ec9f7a-a7b7-45e8-89c2-86497013ea01 None None] MFA Rules not processed for user `69e42a19cb854f63b5dda9e0cf9c0ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:38.242462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-89ec9f7a-a7b7-45e8-89c2-86497013ea01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:38.338991 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1334/2644] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:38 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:38.351707 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9569d8-c515-4925-8f89-9061c97294e8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:38.354414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6c9569d8-c515-4925-8f89-9061c97294e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:38.400522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:38.439640 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9569d8-c515-4925-8f89-9061c97294e8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:38.444956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f837a6b-2db9-4dac-96de-4dbc9de1c857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:38.445877 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f837a6b-2db9-4dac-96de-4dbc9de1c857 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:38.446351 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f837a6b-2db9-4dac-96de-4dbc9de1c857 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:38.446889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f837a6b-2db9-4dac-96de-4dbc9de1c857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:38.458740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f837a6b-2db9-4dac-96de-4dbc9de1c857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:38.461038 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f837a6b-2db9-4dac-96de-4dbc9de1c857 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:38.476925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1311/2645] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:38 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:38.655544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9f7a-a7b7-45e8-89c2-86497013ea01 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:38.665805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9f7a-a7b7-45e8-89c2-86497013ea01 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:38.669683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fe845-4600-4a06-b255-6c4783fa718a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:38.670348 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fe845-4600-4a06-b255-6c4783fa718a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:38.670629 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fe845-4600-4a06-b255-6c4783fa718a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:38.670901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fe845-4600-4a06-b255-6c4783fa718a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:38.712658 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:38.728267 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fe845-4600-4a06-b255-6c4783fa718a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:38.729705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fe845-4600-4a06-b255-6c4783fa718a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:38.739239 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1335/2646] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:38 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:43.516466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f837a6b-2db9-4dac-96de-4dbc9de1c857 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:43.519033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4f837a6b-2db9-4dac-96de-4dbc9de1c857 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:43.609114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f837a6b-2db9-4dac-96de-4dbc9de1c857 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:43.614905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:43.615858 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:43.616139 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:43.616509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] PATH_INFO: `/v3/users/8ff085ab5e6d4d278b72b7b8c528c8d4` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:43.617712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ff085ab5e6d4d278b72b7b8c528c8d4)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:43.619709 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:43.838312 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ff085ab5e6d4d278b72b7b8c528c8d4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:43.846419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ff085ab5e6d4d278b72b7b8c528c8d4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:43.863499 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ff085ab5e6d4d278b72b7b8c528c8d4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:43.926860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fe845-4600-4a06-b255-6c4783fa718a service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:43.939617 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fe845-4600-4a06-b255-6c4783fa718a service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:43.940904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ff085ab5e6d4d278b72b7b8c528c8d4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:43.941679 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ff085ab5e6d4d278b72b7b8c528c8d4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:43.946441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:43.947624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:43.948128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:43.948619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] PATH_INFO: `/v3/users/5ccdd3ea4f6c48b1a2d1b8f6130e91d2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:43.950166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ccdd3ea4f6c48b1a2d1b8f6130e91d2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:43.952613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:43.955708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1312/2647] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:43 2022] DELETE /identity/v3/users/8ff085ab5e6d4d278b72b7b8c528c8d4 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:44.170733 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ccdd3ea4f6c48b1a2d1b8f6130e91d2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:44.178572 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ccdd3ea4f6c48b1a2d1b8f6130e91d2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:44.215514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:44.217148 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:44.256086 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ccdd3ea4f6c48b1a2d1b8f6130e91d2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:44.258052 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ccdd3ea4f6c48b1a2d1b8f6130e91d2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:44.261694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-34ec01cc-d07f-4ac4-8489-c5f77f7aa2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ccdd3ea4f6c48b1a2d1b8f6130e91d2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:44.279930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:44.292978 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1336/2648] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:43 2022] DELETE /identity/v3/users/5ccdd3ea4f6c48b1a2d1b8f6130e91d2 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:44.297767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a2c09-87a5-42d0-8157-548e88de6fe0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:44.298082 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a2c09-87a5-42d0-8157-548e88de6fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:44.298369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a2c09-87a5-42d0-8157-548e88de6fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:44.303660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd949d-6854-4868-9908-c6ae0788e851 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:44.317138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-119a2c09-87a5-42d0-8157-548e88de6fe0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:44.318393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-119a2c09-87a5-42d0-8157-548e88de6fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:44.381240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:44.382387 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:44.382671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:44.382947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] PATH_INFO: `/v3/projects/2362e98f5387452893f972175548189a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:44.384461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2362e98f5387452893f972175548189a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:44.386424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:44.390957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1337/2649] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:14:44.400669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a2c09-87a5-42d0-8157-548e88de6fe0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:44.402171 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-119a2c09-87a5-42d0-8157-548e88de6fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:44.471363 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a2c09-87a5-42d0-8157-548e88de6fe0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:44.475777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ab9ba-8b4f-4cf3-b023-cbc27b8daa6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:44.476439 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ab9ba-8b4f-4cf3-b023-cbc27b8daa6e service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:44.476723 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ab9ba-8b4f-4cf3-b023-cbc27b8daa6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:44.477240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ab9ba-8b4f-4cf3-b023-cbc27b8daa6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:44.488910 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:44.491747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ab9ba-8b4f-4cf3-b023-cbc27b8daa6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:44.493272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ab9ba-8b4f-4cf3-b023-cbc27b8daa6e service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:44.504081 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1338/2650] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:14:44.529291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2362e98f5387452893f972175548189a', 'request_id': 'req-56918f37-816f-4510-8098-522c89a0849a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:44.535725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2362e98f5387452893f972175548189a', 'request_id': 'req-56918f37-816f-4510-8098-522c89a0849a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:44.623928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2362e98f5387452893f972175548189a', 'request_id': 'req-56918f37-816f-4510-8098-522c89a0849a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:44.645304 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] The token cache is being invalidate because project 2362e98f5387452893f972175548189a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:44.645708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:44.646554 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:44.649035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1313/2651] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:44 2022] DELETE /identity/v3/projects/2362e98f5387452893f972175548189a => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:45.092318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ab9ba-8b4f-4cf3-b023-cbc27b8daa6e service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:45.093861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3b4ab9ba-8b4f-4cf3-b023-cbc27b8daa6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:45.108671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ab9ba-8b4f-4cf3-b023-cbc27b8daa6e service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:45.168106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:45.169001 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:45.169284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:45.169564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] PATH_INFO: `/v3/projects/dec2e269c61f4a17bade8696a3693ae6` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:45.170863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dec2e269c61f4a17bade8696a3693ae6)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:45.172785 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:45.263216 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dec2e269c61f4a17bade8696a3693ae6', 'request_id': 'req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:45.268615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dec2e269c61f4a17bade8696a3693ae6', 'request_id': 'req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:45.275601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dec2e269c61f4a17bade8696a3693ae6', 'request_id': 'req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:45.360175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] The token cache is being invalidate because project dec2e269c61f4a17bade8696a3693ae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:45.360463 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:45.361063 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:45.362665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1339/2652] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:45 2022] DELETE /identity/v3/projects/dec2e269c61f4a17bade8696a3693ae6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:45.367171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:45.368634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:45.382024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56918f37-816f-4510-8098-522c89a0849a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:45.436983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:45.437838 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:45.438140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:45.438410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] PATH_INFO: `/v3/users/69e42a19cb854f63b5dda9e0cf9c0ec4` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:45.439571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e42a19cb854f63b5dda9e0cf9c0ec4)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:45.442059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:45.463446 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:45.661264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e42a19cb854f63b5dda9e0cf9c0ec4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:45.668406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e42a19cb854f63b5dda9e0cf9c0ec4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:45.681228 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e42a19cb854f63b5dda9e0cf9c0ec4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:45.738368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e42a19cb854f63b5dda9e0cf9c0ec4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:45.739125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e1452fea-647c-47e8-b83b-594554e6583c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e42a19cb854f63b5dda9e0cf9c0ec4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:45.746506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1314/2653] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:45 2022] DELETE /identity/v3/users/69e42a19cb854f63b5dda9e0cf9c0ec4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:46.168485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:46.170126 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:46.185164 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ee36f-4159-4cbe-9b23-5cd2edf03d5a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:46.244564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:46.245339 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:46.245618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:46.245891 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] PATH_INFO: `/v3/projects/7b1a60bae9e84c9d9da7c564af7fb0d0` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:46.247237 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1a60bae9e84c9d9da7c564af7fb0d0)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:46.249143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:46.318162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:46.332345 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1a60bae9e84c9d9da7c564af7fb0d0', 'request_id': 'req-3d7971e0-49a1-485d-8696-45e96f356229'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:46.338479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1a60bae9e84c9d9da7c564af7fb0d0', 'request_id': 'req-3d7971e0-49a1-485d-8696-45e96f356229'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:46.345057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1a60bae9e84c9d9da7c564af7fb0d0', 'request_id': 'req-3d7971e0-49a1-485d-8696-45e96f356229'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:46.388241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] The token cache is being invalidate because project 7b1a60bae9e84c9d9da7c564af7fb0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:46.388525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:46.389135 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3d7971e0-49a1-485d-8696-45e96f356229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:46.390721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1340/2654] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:46 2022] DELETE /identity/v3/projects/7b1a60bae9e84c9d9da7c564af7fb0d0 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:46.396163 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2d5e-196f-4222-a8f0-c8fd3e5c0114 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:46.396615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2d5e-196f-4222-a8f0-c8fd3e5c0114 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:46.396753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe2d5e-196f-4222-a8f0-c8fd3e5c0114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:46.415018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-cdbe2d5e-196f-4222-a8f0-c8fd3e5c0114 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:46.416210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cdbe2d5e-196f-4222-a8f0-c8fd3e5c0114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:46.474728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1315/2655] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:14:46.483482 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7371-e783-430f-b2e8-9db7a82568b5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:46.483775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7371-e783-430f-b2e8-9db7a82568b5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:46.484060 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7371-e783-430f-b2e8-9db7a82568b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:46.502775 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a11a7371-e783-430f-b2e8-9db7a82568b5 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:46.503922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a11a7371-e783-430f-b2e8-9db7a82568b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:46.512164 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1341/2656] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:46.516346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe2d5e-196f-4222-a8f0-c8fd3e5c0114 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:46.526166 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe2d5e-196f-4222-a8f0-c8fd3e5c0114 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:46.530677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c991fe-1ace-468c-86f3-2fb0ffffd2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:46.531323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c991fe-1ace-468c-86f3-2fb0ffffd2ac admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:46.531605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c991fe-1ace-468c-86f3-2fb0ffffd2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:46.531881 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c991fe-1ace-468c-86f3-2fb0ffffd2ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:46.532601 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c991fe-1ace-468c-86f3-2fb0ffffd2ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:14:46.532953 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c991fe-1ace-468c-86f3-2fb0ffffd2ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:46.534582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c991fe-1ace-468c-86f3-2fb0ffffd2ac admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:46.546566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1316/2657] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:14:46.550819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a7371-e783-430f-b2e8-9db7a82568b5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:46.560611 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a7371-e783-430f-b2e8-9db7a82568b5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:46.565317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c738878-0173-465c-bf4d-58d67edbc388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:46.566131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c738878-0173-465c-bf4d-58d67edbc388 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:46.566412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c738878-0173-465c-bf4d-58d67edbc388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:46.566683 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c738878-0173-465c-bf4d-58d67edbc388 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:46.567386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c738878-0173-465c-bf4d-58d67edbc388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:46.569247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c738878-0173-465c-bf4d-58d67edbc388 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:46.652939 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1342/2658] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:14:46 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:46.658187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c991fe-1ace-468c-86f3-2fb0ffffd2ac admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:46.669284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c991fe-1ace-468c-86f3-2fb0ffffd2ac admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:46.725985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70072ad9-f872-4dd7-8efd-f43dafdfd89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:46.726667 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70072ad9-f872-4dd7-8efd-f43dafdfd89d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:46.726944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70072ad9-f872-4dd7-8efd-f43dafdfd89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:46.727218 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70072ad9-f872-4dd7-8efd-f43dafdfd89d admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:46.727889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70072ad9-f872-4dd7-8efd-f43dafdfd89d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:46.729768 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70072ad9-f872-4dd7-8efd-f43dafdfd89d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:46.818293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1317/2659] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:14:46 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:46.822258 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c738878-0173-465c-bf4d-58d67edbc388 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:46.833483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c738878-0173-465c-bf4d-58d67edbc388 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:46.838360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c65ec-6cc6-435e-b937-5a315c52386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:46.839130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c65ec-6cc6-435e-b937-5a315c52386f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:46.839415 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c65ec-6cc6-435e-b937-5a315c52386f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:46.839692 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c65ec-6cc6-435e-b937-5a315c52386f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:46.840414 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c65ec-6cc6-435e-b937-5a315c52386f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:46.842006 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c65ec-6cc6-435e-b937-5a315c52386f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:46.853579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1343/2660] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:46.857884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70072ad9-f872-4dd7-8efd-f43dafdfd89d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:46.868388 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70072ad9-f872-4dd7-8efd-f43dafdfd89d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:46.873344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b77edd-45f1-4805-970c-b4107bc4b1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:46.874020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b77edd-45f1-4805-970c-b4107bc4b1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:46.874320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b77edd-45f1-4805-970c-b4107bc4b1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:46.874598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b77edd-45f1-4805-970c-b4107bc4b1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:46.875328 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b77edd-45f1-4805-970c-b4107bc4b1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:46.877160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b77edd-45f1-4805-970c-b4107bc4b1c3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:46.890107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1318/2661] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:46.894412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c65ec-6cc6-435e-b937-5a315c52386f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:46.905938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c65ec-6cc6-435e-b937-5a315c52386f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:46.910620 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1c09-ccf4-42de-b5d2-611c6c224f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:46.912074 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1c09-ccf4-42de-b5d2-611c6c224f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:46.912360 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1c09-ccf4-42de-b5d2-611c6c224f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:46.912634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1c09-ccf4-42de-b5d2-611c6c224f1d admin admin] PATH_INFO: `/v3/projects/d29fb8e83c6445bbb144459847ffcbaf/users/60f9c74413e24319accd7840b2e07060/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:46.927506 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1c09-ccf4-42de-b5d2-611c6c224f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29fb8e83c6445bbb144459847ffcbaf, user_id=60f9c74413e24319accd7840b2e07060, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:46.930178 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1c09-ccf4-42de-b5d2-611c6c224f1d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:47.007622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1344/2662] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:46 2022] PUT /identity/v3/projects/d29fb8e83c6445bbb144459847ffcbaf/users/60f9c74413e24319accd7840b2e07060/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:47.017983 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135223a-4e65-44e2-b721-f22336a4009b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:47.018222 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135223a-4e65-44e2-b721-f22336a4009b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:47.018505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135223a-4e65-44e2-b721-f22336a4009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:47.032594 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:47.048555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b135223a-4e65-44e2-b721-f22336a4009b None None] MFA Rules not processed for user `60f9c74413e24319accd7840b2e07060`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:47.049742 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b135223a-4e65-44e2-b721-f22336a4009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:47.119064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1319/2663] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:47.123853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1c09-ccf4-42de-b5d2-611c6c224f1d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:47.188449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1c09-ccf4-42de-b5d2-611c6c224f1d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:47.196515 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298469d1-b9c4-422c-92cb-fc1cc0c9bc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:47.197181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298469d1-b9c4-422c-92cb-fc1cc0c9bc3d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:47.197467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298469d1-b9c4-422c-92cb-fc1cc0c9bc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:47.197748 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298469d1-b9c4-422c-92cb-fc1cc0c9bc3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:47.198485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298469d1-b9c4-422c-92cb-fc1cc0c9bc3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:47.200470 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298469d1-b9c4-422c-92cb-fc1cc0c9bc3d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:47.287861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1345/2664] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:14:47 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:47.291595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135223a-4e65-44e2-b721-f22336a4009b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:47.302567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135223a-4e65-44e2-b721-f22336a4009b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:47.359246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d54e19-60d2-43ea-bd55-73f53278dfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:47.359917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d54e19-60d2-43ea-bd55-73f53278dfb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:47.360202 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d54e19-60d2-43ea-bd55-73f53278dfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:47.360497 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d54e19-60d2-43ea-bd55-73f53278dfb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:47.361175 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d54e19-60d2-43ea-bd55-73f53278dfb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:47.363152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d54e19-60d2-43ea-bd55-73f53278dfb7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:47.452136 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1320/2665] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:14:47 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:47.456096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298469d1-b9c4-422c-92cb-fc1cc0c9bc3d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:47.467056 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298469d1-b9c4-422c-92cb-fc1cc0c9bc3d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:47.471681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc895f03-0e83-4992-b434-4a0d47e67223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:47.472331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc895f03-0e83-4992-b434-4a0d47e67223 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:47.472693 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc895f03-0e83-4992-b434-4a0d47e67223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:47.472968 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc895f03-0e83-4992-b434-4a0d47e67223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:47.473843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc895f03-0e83-4992-b434-4a0d47e67223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:47.475714 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc895f03-0e83-4992-b434-4a0d47e67223 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:47.487227 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1346/2666] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:47.491505 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d54e19-60d2-43ea-bd55-73f53278dfb7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:47.501300 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d54e19-60d2-43ea-bd55-73f53278dfb7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:47.507027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f18e2a-adc5-4118-894d-39bf8895d69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:47.507897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f18e2a-adc5-4118-894d-39bf8895d69f admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:47.508217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f18e2a-adc5-4118-894d-39bf8895d69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:47.508489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f18e2a-adc5-4118-894d-39bf8895d69f admin admin] PATH_INFO: `/v3/projects/231049cd60894cada670c399de92c944/users/d644638bf8824cfcb96690fcb0ffb184/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:47.524074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f18e2a-adc5-4118-894d-39bf8895d69f admin admin] RBAC: Authorizing `identity:create_grant(project_id=231049cd60894cada670c399de92c944, user_id=d644638bf8824cfcb96690fcb0ffb184, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:47.526802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f18e2a-adc5-4118-894d-39bf8895d69f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:47.618058 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1321/2667] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:47 2022] PUT /identity/v3/projects/231049cd60894cada670c399de92c944/users/d644638bf8824cfcb96690fcb0ffb184/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:47.622185 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc895f03-0e83-4992-b434-4a0d47e67223 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:47.632533 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc895f03-0e83-4992-b434-4a0d47e67223 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:47.656434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:47.688936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d180f-9258-4f45-8b63-9d3552c78c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:47.689597 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d180f-9258-4f45-8b63-9d3552c78c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:47.689886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d180f-9258-4f45-8b63-9d3552c78c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:47.690215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d180f-9258-4f45-8b63-9d3552c78c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:47.691018 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d180f-9258-4f45-8b63-9d3552c78c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:47.692784 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d180f-9258-4f45-8b63-9d3552c78c1c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:47.702763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1347/2668] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:47.706929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f18e2a-adc5-4118-894d-39bf8895d69f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:47.717891 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f18e2a-adc5-4118-894d-39bf8895d69f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:47.722476 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768d752-a4e0-43c6-8790-9480bdc1f880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:47.723349 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768d752-a4e0-43c6-8790-9480bdc1f880 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:47.724955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768d752-a4e0-43c6-8790-9480bdc1f880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:47.724955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768d752-a4e0-43c6-8790-9480bdc1f880 admin admin] PATH_INFO: `/v3/projects/231049cd60894cada670c399de92c944/users/d644638bf8824cfcb96690fcb0ffb184/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:47.726487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768d752-a4e0-43c6-8790-9480bdc1f880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231049cd60894cada670c399de92c944, user_id=d644638bf8824cfcb96690fcb0ffb184, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:47.729108 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768d752-a4e0-43c6-8790-9480bdc1f880 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:47.739303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1322/2669] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:47 2022] PUT /identity/v3/projects/231049cd60894cada670c399de92c944/users/d644638bf8824cfcb96690fcb0ffb184/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:47.750151 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9e38f-05c1-4f77-8cef-363d960f757a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:47.751614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9e38f-05c1-4f77-8cef-363d960f757a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:47.751614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9e38f-05c1-4f77-8cef-363d960f757a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:47.779304 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f6e9e38f-05c1-4f77-8cef-363d960f757a None None] MFA Rules not processed for user `d644638bf8824cfcb96690fcb0ffb184`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:47.780501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f6e9e38f-05c1-4f77-8cef-363d960f757a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:47.849553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1348/2670] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:47.860850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768d752-a4e0-43c6-8790-9480bdc1f880 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:47.862538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8768d752-a4e0-43c6-8790-9480bdc1f880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:47.916251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:47.919808 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768d752-a4e0-43c6-8790-9480bdc1f880 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:47.923468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d831c-6de8-495f-a349-c1d57a450cee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:47.924106 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d831c-6de8-495f-a349-c1d57a450cee service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:47.924416 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d831c-6de8-495f-a349-c1d57a450cee service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:47.924690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d831c-6de8-495f-a349-c1d57a450cee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:47.933684 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d831c-6de8-495f-a349-c1d57a450cee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:47.935167 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d831c-6de8-495f-a349-c1d57a450cee service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:47.945000 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1323/2671] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:47.972635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9e38f-05c1-4f77-8cef-363d960f757a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:48.037405 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9e38f-05c1-4f77-8cef-363d960f757a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:48.042277 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:48.043154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:48.043434 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:48.043706 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] PATH_INFO: `/v3/users/60f9c74413e24319accd7840b2e07060` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:48.044990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60f9c74413e24319accd7840b2e07060)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:48.046941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:48.249875 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60f9c74413e24319accd7840b2e07060'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:48.252531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:48.257656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60f9c74413e24319accd7840b2e07060'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:48.320739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60f9c74413e24319accd7840b2e07060'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:48.321423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60f9c74413e24319accd7840b2e07060'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:48.322057 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60f9c74413e24319accd7840b2e07060'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:48.348375 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1349/2672] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:47 2022] DELETE /identity/v3/users/60f9c74413e24319accd7840b2e07060 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:48.353235 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec2448-09ad-4681-9975-66d50bb46717 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:48.353595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec2448-09ad-4681-9975-66d50bb46717 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:48.353929 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec2448-09ad-4681-9975-66d50bb46717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:48.371706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-caec2448-09ad-4681-9975-66d50bb46717 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:48.372899 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-caec2448-09ad-4681-9975-66d50bb46717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:48.432889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1324/2673] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:48.442606 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:48.444029 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:48.502642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cdee3-6a87-440e-9cdc-e6b77a721011 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:48.506835 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6e3da-67ed-4712-875d-1db79b604572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:48.507478 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6e3da-67ed-4712-875d-1db79b604572 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:48.507761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6e3da-67ed-4712-875d-1db79b604572 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:48.508034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6e3da-67ed-4712-875d-1db79b604572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:48.517174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6e3da-67ed-4712-875d-1db79b604572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:48.518690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6e3da-67ed-4712-875d-1db79b604572 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:48.528672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1350/2674] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:48.916627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec2448-09ad-4681-9975-66d50bb46717 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:48.918151 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-caec2448-09ad-4681-9975-66d50bb46717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:48.930344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec2448-09ad-4681-9975-66d50bb46717 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:48.935354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:48.936221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:48.936500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:48.936776 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] PATH_INFO: `/v3/projects/d29fb8e83c6445bbb144459847ffcbaf` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:48.941730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d29fb8e83c6445bbb144459847ffcbaf)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:48.941730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:49.075829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd29fb8e83c6445bbb144459847ffcbaf', 'request_id': 'req-3fe1fc3b-ba6f-472d-8797-05ef97f95132'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:49.081311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd29fb8e83c6445bbb144459847ffcbaf', 'request_id': 'req-3fe1fc3b-ba6f-472d-8797-05ef97f95132'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:49.133334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd29fb8e83c6445bbb144459847ffcbaf', 'request_id': 'req-3fe1fc3b-ba6f-472d-8797-05ef97f95132'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:49.144657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] The token cache is being invalidate because project d29fb8e83c6445bbb144459847ffcbaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:49.144943 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:49.145568 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:49.147255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1325/2675] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:48 2022] DELETE /identity/v3/projects/d29fb8e83c6445bbb144459847ffcbaf => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:49.151204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6e3da-67ed-4712-875d-1db79b604572 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:49.160766 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6e3da-67ed-4712-875d-1db79b604572 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:49.217803 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:49.218886 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:49.219173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:49.219447 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] PATH_INFO: `/v3/users/d644638bf8824cfcb96690fcb0ffb184` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:49.220669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d644638bf8824cfcb96690fcb0ffb184)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:49.222651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:49.404017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd644638bf8824cfcb96690fcb0ffb184'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:49.411479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd644638bf8824cfcb96690fcb0ffb184'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:49.481446 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd644638bf8824cfcb96690fcb0ffb184'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:49.482102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd644638bf8824cfcb96690fcb0ffb184'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:49.482716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e2be291a-fee8-4f59-8c87-1cd3b490ba60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd644638bf8824cfcb96690fcb0ffb184'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:49.492357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:49.507598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1351/2676] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:49 2022] DELETE /identity/v3/users/d644638bf8824cfcb96690fcb0ffb184 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:49.921704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:49.923230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:49.937001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1fc3b-ba6f-472d-8797-05ef97f95132 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:49.995695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:49.996562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:49.996844 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:49.997117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] PATH_INFO: `/v3/projects/231049cd60894cada670c399de92c944` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:49.998372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] RBAC: Authorizing `identity:delete_project(project_id=231049cd60894cada670c399de92c944)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:50.000275 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:50.018315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:50.131958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231049cd60894cada670c399de92c944', 'request_id': 'req-9a7d4d30-9803-4893-965e-8b58c3d97ead'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:50.137179 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231049cd60894cada670c399de92c944', 'request_id': 'req-9a7d4d30-9803-4893-965e-8b58c3d97ead'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:50.252941 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231049cd60894cada670c399de92c944', 'request_id': 'req-9a7d4d30-9803-4893-965e-8b58c3d97ead'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:50.265429 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] The token cache is being invalidate because project 231049cd60894cada670c399de92c944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:50.269584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:50.269584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9a7d4d30-9803-4893-965e-8b58c3d97ead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:50.269584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1326/2677] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:49 2022] DELETE /identity/v3/projects/231049cd60894cada670c399de92c944 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:50.273640 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b6e1a-10a1-4c3e-a892-bb1bbe6c6217 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:50.274665 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b6e1a-10a1-4c3e-a892-bb1bbe6c6217 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:50.274976 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b6e1a-10a1-4c3e-a892-bb1bbe6c6217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:50.292704 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4c4b6e1a-10a1-4c3e-a892-bb1bbe6c6217 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:50.293912 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4c4b6e1a-10a1-4c3e-a892-bb1bbe6c6217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:50.362551 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1352/2678] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:50.370583 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cffff7-7584-4177-99d5-a50f49f76c90 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:50.370930 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cffff7-7584-4177-99d5-a50f49f76c90 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:50.371207 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cffff7-7584-4177-99d5-a50f49f76c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:50.392160 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-18cffff7-7584-4177-99d5-a50f49f76c90 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:50.393373 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-18cffff7-7584-4177-99d5-a50f49f76c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:50.402567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1327/2679] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:50.406438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b6e1a-10a1-4c3e-a892-bb1bbe6c6217 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:50.417618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b6e1a-10a1-4c3e-a892-bb1bbe6c6217 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:50.422400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5afef2-913a-427e-a04c-6ae3a0ae199e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:50.423073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5afef2-913a-427e-a04c-6ae3a0ae199e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:50.423354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5afef2-913a-427e-a04c-6ae3a0ae199e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:50.423636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5afef2-913a-427e-a04c-6ae3a0ae199e admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:50.424357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5afef2-913a-427e-a04c-6ae3a0ae199e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:14:50.424850 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5afef2-913a-427e-a04c-6ae3a0ae199e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:50.426310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5afef2-913a-427e-a04c-6ae3a0ae199e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:50.443212 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1353/2680] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:14:50.444850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cffff7-7584-4177-99d5-a50f49f76c90 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:50.454598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cffff7-7584-4177-99d5-a50f49f76c90 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:50.462748 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5c190-6cce-49d9-8c43-426d3a1ca6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:50.463424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5c190-6cce-49d9-8c43-426d3a1ca6bc admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:50.463706 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5c190-6cce-49d9-8c43-426d3a1ca6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:50.463980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5c190-6cce-49d9-8c43-426d3a1ca6bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:50.464677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5c190-6cce-49d9-8c43-426d3a1ca6bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:50.466740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5c190-6cce-49d9-8c43-426d3a1ca6bc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:50.577889 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1328/2681] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:14:50 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:50.579233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5afef2-913a-427e-a04c-6ae3a0ae199e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:50.590087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5afef2-913a-427e-a04c-6ae3a0ae199e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:50.650512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2f977-6a9b-4cea-a5d6-0735e620374f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:50.651195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2f977-6a9b-4cea-a5d6-0735e620374f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:50.651473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2f977-6a9b-4cea-a5d6-0735e620374f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:50.651753 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2f977-6a9b-4cea-a5d6-0735e620374f admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:50.652474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2f977-6a9b-4cea-a5d6-0735e620374f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:50.654438 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2f977-6a9b-4cea-a5d6-0735e620374f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:50.721179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1354/2682] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:14:50 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:50.728142 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5c190-6cce-49d9-8c43-426d3a1ca6bc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:50.738319 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5c190-6cce-49d9-8c43-426d3a1ca6bc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:50.743949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf3422-b783-448c-813e-d3d2486d66b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:50.744615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3422-b783-448c-813e-d3d2486d66b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:50.744887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3422-b783-448c-813e-d3d2486d66b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:50.745161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3422-b783-448c-813e-d3d2486d66b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:50.745878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf3422-b783-448c-813e-d3d2486d66b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:50.747507 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf3422-b783-448c-813e-d3d2486d66b6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:50.758564 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1329/2683] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:50.762412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2f977-6a9b-4cea-a5d6-0735e620374f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:50.773186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2f977-6a9b-4cea-a5d6-0735e620374f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:50.777786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0368a-8207-44e5-86b3-7465e5b65440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:50.778473 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0368a-8207-44e5-86b3-7465e5b65440 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:50.778756 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0368a-8207-44e5-86b3-7465e5b65440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:50.779058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0368a-8207-44e5-86b3-7465e5b65440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:50.779774 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0368a-8207-44e5-86b3-7465e5b65440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:50.781354 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0368a-8207-44e5-86b3-7465e5b65440 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:50.791485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1355/2684] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:50.795486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf3422-b783-448c-813e-d3d2486d66b6 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:50.806315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf3422-b783-448c-813e-d3d2486d66b6 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:50.811122 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e3bf9-0f0a-4d11-9593-83d707a13024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:50.811915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e3bf9-0f0a-4d11-9593-83d707a13024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:50.812196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e3bf9-0f0a-4d11-9593-83d707a13024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:50.812472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e3bf9-0f0a-4d11-9593-83d707a13024 admin admin] PATH_INFO: `/v3/projects/f6d33a805c3b44309b3608832355a075/users/7f99a5e1a3d64f2d815276a5d6e07243/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:50.828165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e3bf9-0f0a-4d11-9593-83d707a13024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6d33a805c3b44309b3608832355a075, user_id=7f99a5e1a3d64f2d815276a5d6e07243, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:50.830869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e3bf9-0f0a-4d11-9593-83d707a13024 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:50.897962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1330/2685] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:50 2022] PUT /identity/v3/projects/f6d33a805c3b44309b3608832355a075/users/7f99a5e1a3d64f2d815276a5d6e07243/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:50.908248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26b5bf-e575-485f-bf40-804a584989b3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:50.908562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26b5bf-e575-485f-bf40-804a584989b3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:50.908843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26b5bf-e575-485f-bf40-804a584989b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:50.936590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bd26b5bf-e575-485f-bf40-804a584989b3 None None] MFA Rules not processed for user `7f99a5e1a3d64f2d815276a5d6e07243`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:50.937777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bd26b5bf-e575-485f-bf40-804a584989b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:50.982146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:51.003317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1356/2686] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:51.009630 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e3bf9-0f0a-4d11-9593-83d707a13024 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:51.071353 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e3bf9-0f0a-4d11-9593-83d707a13024 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:51.076018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aa691-8fa7-49f6-b070-e9ddda5150c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:51.076675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa691-8fa7-49f6-b070-e9ddda5150c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:51.076960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa691-8fa7-49f6-b070-e9ddda5150c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:51.077238 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa691-8fa7-49f6-b070-e9ddda5150c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:51.077924 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aa691-8fa7-49f6-b070-e9ddda5150c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:51.079868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aa691-8fa7-49f6-b070-e9ddda5150c1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:51.150274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1331/2687] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:14:51 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:51.154249 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26b5bf-e575-485f-bf40-804a584989b3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:51.165788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26b5bf-e575-485f-bf40-804a584989b3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:51.221646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5fd65-ec1b-4014-943f-edd51d888848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:51.222335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5fd65-ec1b-4014-943f-edd51d888848 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:51.222621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5fd65-ec1b-4014-943f-edd51d888848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:51.222901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5fd65-ec1b-4014-943f-edd51d888848 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:51.223596 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5fd65-ec1b-4014-943f-edd51d888848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:51.226174 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5fd65-ec1b-4014-943f-edd51d888848 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:51.286191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1357/2688] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:14:51 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:51.290803 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aa691-8fa7-49f6-b070-e9ddda5150c1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:51.301325 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aa691-8fa7-49f6-b070-e9ddda5150c1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:51.306228 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261638e-eac7-4303-b309-b0f770a08925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:51.306884 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261638e-eac7-4303-b309-b0f770a08925 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:51.307165 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261638e-eac7-4303-b309-b0f770a08925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:51.307440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261638e-eac7-4303-b309-b0f770a08925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:51.308215 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261638e-eac7-4303-b309-b0f770a08925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:51.309795 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261638e-eac7-4303-b309-b0f770a08925 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:51.320378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1332/2689] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:51.324386 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5fd65-ec1b-4014-943f-edd51d888848 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:51.334853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5fd65-ec1b-4014-943f-edd51d888848 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:51.339542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f2139-d4b6-40a9-91f9-302c917f9357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:51.340422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f2139-d4b6-40a9-91f9-302c917f9357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:51.340700 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f2139-d4b6-40a9-91f9-302c917f9357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:51.340975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f2139-d4b6-40a9-91f9-302c917f9357 admin admin] PATH_INFO: `/v3/projects/c374b7c6f49f460bbe03e572d503c77a/users/53c4d87665cb49e18aff95012dd33eb6/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:51.357512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f2139-d4b6-40a9-91f9-302c917f9357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c374b7c6f49f460bbe03e572d503c77a, user_id=53c4d87665cb49e18aff95012dd33eb6, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:51.360170 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f2139-d4b6-40a9-91f9-302c917f9357 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:51.418612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1358/2690] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:51 2022] PUT /identity/v3/projects/c374b7c6f49f460bbe03e572d503c77a/users/53c4d87665cb49e18aff95012dd33eb6/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:51.422551 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261638e-eac7-4303-b309-b0f770a08925 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:51.432580 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261638e-eac7-4303-b309-b0f770a08925 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:51.443739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:51.488020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec790f6d-879c-4a05-a62b-eff523e00887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:51.488656 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec790f6d-879c-4a05-a62b-eff523e00887 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:51.488942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec790f6d-879c-4a05-a62b-eff523e00887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:51.489208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec790f6d-879c-4a05-a62b-eff523e00887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:51.489914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec790f6d-879c-4a05-a62b-eff523e00887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:51.491623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec790f6d-879c-4a05-a62b-eff523e00887 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:51.501625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1333/2691] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:51.505889 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f2139-d4b6-40a9-91f9-302c917f9357 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:51.516538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f2139-d4b6-40a9-91f9-302c917f9357 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:51.521282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f0982-df63-477b-8430-40306319309c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:51.522841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f0982-df63-477b-8430-40306319309c admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:51.523145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f0982-df63-477b-8430-40306319309c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:51.523422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f0982-df63-477b-8430-40306319309c admin admin] PATH_INFO: `/v3/projects/c374b7c6f49f460bbe03e572d503c77a/users/53c4d87665cb49e18aff95012dd33eb6/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:51.525407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f0982-df63-477b-8430-40306319309c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c374b7c6f49f460bbe03e572d503c77a, user_id=53c4d87665cb49e18aff95012dd33eb6, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:51.528062 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f0982-df63-477b-8430-40306319309c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:51.539255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1359/2692] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:51 2022] PUT /identity/v3/projects/c374b7c6f49f460bbe03e572d503c77a/users/53c4d87665cb49e18aff95012dd33eb6/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:14:51.550414 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee784ca-a617-4e93-acac-3054d35ebd72 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:51.550767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee784ca-a617-4e93-acac-3054d35ebd72 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:51.551110 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee784ca-a617-4e93-acac-3054d35ebd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:51.579155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5ee784ca-a617-4e93-acac-3054d35ebd72 None None] MFA Rules not processed for user `53c4d87665cb49e18aff95012dd33eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:51.580407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5ee784ca-a617-4e93-acac-3054d35ebd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:51.653955 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1334/2693] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:51.666316 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f0982-df63-477b-8430-40306319309c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:51.668058 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-666f0982-df63-477b-8430-40306319309c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:51.727069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f0982-df63-477b-8430-40306319309c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:51.730938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bebf1a-92cc-4730-9fef-5a6f66f469c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:51.735112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bebf1a-92cc-4730-9fef-5a6f66f469c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:51.735112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bebf1a-92cc-4730-9fef-5a6f66f469c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:51.735112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bebf1a-92cc-4730-9fef-5a6f66f469c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:51.742826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bebf1a-92cc-4730-9fef-5a6f66f469c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:51.744286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bebf1a-92cc-4730-9fef-5a6f66f469c2 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:51.755014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1360/2694] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:52.812710 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee784ca-a617-4e93-acac-3054d35ebd72 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:52.822965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee784ca-a617-4e93-acac-3054d35ebd72 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:52.828625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30002409-b5aa-40b5-a644-733acad5931b tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53c4d87665cb49e18aff95012dd33eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c374b7c6f49f460bbe03e572d503c77a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:52.834205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30002409-b5aa-40b5-a644-733acad5931b tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:52.834205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30002409-b5aa-40b5-a644-733acad5931b tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:52.834205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30002409-b5aa-40b5-a644-733acad5931b tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:52.834205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30002409-b5aa-40b5-a644-733acad5931b tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:14:52.834205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30002409-b5aa-40b5-a644-733acad5931b tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:52.835132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30002409-b5aa-40b5-a644-733acad5931b tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:52.846326 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1335/2695] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:14:52.850281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bebf1a-92cc-4730-9fef-5a6f66f469c2 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:52.860022 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bebf1a-92cc-4730-9fef-5a6f66f469c2 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:52.864768 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9f298-ff13-4a7c-9b55-80064760482e tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53c4d87665cb49e18aff95012dd33eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c374b7c6f49f460bbe03e572d503c77a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:52.865416 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9f298-ff13-4a7c-9b55-80064760482e tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:52.865711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9f298-ff13-4a7c-9b55-80064760482e tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:52.865980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9f298-ff13-4a7c-9b55-80064760482e tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:52.866720 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9f298-ff13-4a7c-9b55-80064760482e tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:52.869201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9f298-ff13-4a7c-9b55-80064760482e tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:52.972754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1361/2696] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:14:52 2022] POST /identity/v3/projects => generated 354 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:14:52.976782 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30002409-b5aa-40b5-a644-733acad5931b tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:52.987157 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30002409-b5aa-40b5-a644-733acad5931b tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:53.047374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1378d21-7762-40a3-897b-0b845c413984 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53c4d87665cb49e18aff95012dd33eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c374b7c6f49f460bbe03e572d503c77a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:53.048041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1378d21-7762-40a3-897b-0b845c413984 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:53.048334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1378d21-7762-40a3-897b-0b845c413984 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:53.048614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1378d21-7762-40a3-897b-0b845c413984 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:53.049334 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1378d21-7762-40a3-897b-0b845c413984 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:14:53.049644 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1378d21-7762-40a3-897b-0b845c413984 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:53.051265 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1378d21-7762-40a3-897b-0b845c413984 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:53.063335 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1336/2697] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:14:53.099871 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9f298-ff13-4a7c-9b55-80064760482e tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:53.111010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9f298-ff13-4a7c-9b55-80064760482e tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:53.115475 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53c4d87665cb49e18aff95012dd33eb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c374b7c6f49f460bbe03e572d503c77a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:53.116310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:53.116590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:53.116861 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] PATH_INFO: `/v3/projects/97ed06501c3b476eac5e122d0cc0ec0a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:53.119177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: Authorizing `identity:delete_project(project_id=97ed06501c3b476eac5e122d0cc0ec0a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:53.121072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:53.132929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:53.240819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ed06501c3b476eac5e122d0cc0ec0a', 'request_id': 'req-e44c76c1-70d6-4f19-8996-36c6f2b44104'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:53.246797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ed06501c3b476eac5e122d0cc0ec0a', 'request_id': 'req-e44c76c1-70d6-4f19-8996-36c6f2b44104'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:53.252883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ed06501c3b476eac5e122d0cc0ec0a', 'request_id': 'req-e44c76c1-70d6-4f19-8996-36c6f2b44104'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:53.349376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] The token cache is being invalidate because project 97ed06501c3b476eac5e122d0cc0ec0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:53.349669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:53.354795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e44c76c1-70d6-4f19-8996-36c6f2b44104 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:53.354795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1362/2698] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:53 2022] DELETE /identity/v3/projects/97ed06501c3b476eac5e122d0cc0ec0a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:53.939767 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1378d21-7762-40a3-897b-0b845c413984 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:53.941895 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f1378d21-7762-40a3-897b-0b845c413984 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:53.971043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:54.019543 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1378d21-7762-40a3-897b-0b845c413984 tempest-VolumeQuotaClassesTest-1103164615 tempest-VolumeQuotaClassesTest-1103164615-project-admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:54.079399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:54.080010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:54.080293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:54.080569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] PATH_INFO: `/v3/users/7f99a5e1a3d64f2d815276a5d6e07243` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:54.082133 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f99a5e1a3d64f2d815276a5d6e07243)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:54.084061 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:54.230023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f99a5e1a3d64f2d815276a5d6e07243'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:54.238721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f99a5e1a3d64f2d815276a5d6e07243'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:54.257332 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f99a5e1a3d64f2d815276a5d6e07243'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:54.326729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f99a5e1a3d64f2d815276a5d6e07243'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:54.327347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f99a5e1a3d64f2d815276a5d6e07243'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:54.333753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1337/2699] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:53 2022] DELETE /identity/v3/users/7f99a5e1a3d64f2d815276a5d6e07243 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:54.339136 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d5ac8-8f29-47b9-8fb3-610609c5c1cd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:54.339653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d5ac8-8f29-47b9-8fb3-610609c5c1cd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:54.339798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d5ac8-8f29-47b9-8fb3-610609c5c1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:54.366247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9a8d5ac8-8f29-47b9-8fb3-610609c5c1cd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:54.367452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9a8d5ac8-8f29-47b9-8fb3-610609c5c1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:54.426426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1363/2700] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:54.436689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:54.438176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:54.486995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:54.502569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60f083-32d3-431c-871d-dfb7eafc7bbd admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:54.503020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d55ee-f04d-4b60-972a-35fb9014daa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:54.503659 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d55ee-f04d-4b60-972a-35fb9014daa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:54.503937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d55ee-f04d-4b60-972a-35fb9014daa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:54.504208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d55ee-f04d-4b60-972a-35fb9014daa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:54.514698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d55ee-f04d-4b60-972a-35fb9014daa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:54.516130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d55ee-f04d-4b60-972a-35fb9014daa9 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:54.526279 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1338/2701] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:54.983657 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d5ac8-8f29-47b9-8fb3-610609c5c1cd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:54.985145 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9a8d5ac8-8f29-47b9-8fb3-610609c5c1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:54.998787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d5ac8-8f29-47b9-8fb3-610609c5c1cd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:55.004107 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:55.004990 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:55.005268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:55.005537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] PATH_INFO: `/v3/projects/f6d33a805c3b44309b3608832355a075` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:55.007102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6d33a805c3b44309b3608832355a075)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:55.009909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:55.126087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6d33a805c3b44309b3608832355a075', 'request_id': 'req-bd9f67b3-2f93-4371-873f-b21361575ca4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:55.131849 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6d33a805c3b44309b3608832355a075', 'request_id': 'req-bd9f67b3-2f93-4371-873f-b21361575ca4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:55.138372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6d33a805c3b44309b3608832355a075', 'request_id': 'req-bd9f67b3-2f93-4371-873f-b21361575ca4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:55.217358 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] The token cache is being invalidate because project f6d33a805c3b44309b3608832355a075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:55.217653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:55.218320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:55.219862 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1364/2702] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:54 2022] DELETE /identity/v3/projects/f6d33a805c3b44309b3608832355a075 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:55.223699 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d55ee-f04d-4b60-972a-35fb9014daa9 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:55.234327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d55ee-f04d-4b60-972a-35fb9014daa9 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:55.297737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:55.298731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:55.299105 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:55.299453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] PATH_INFO: `/v3/users/53c4d87665cb49e18aff95012dd33eb6` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:55.300739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c4d87665cb49e18aff95012dd33eb6)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:55.302813 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:55.426825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:55.432435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:55.461451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:55.490333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f67b3-2f93-4371-873f-b21361575ca4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:55.494732 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dceb2-c9db-458a-b497-9a820102e2f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:55.495381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dceb2-c9db-458a-b497-9a820102e2f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:55.495664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dceb2-c9db-458a-b497-9a820102e2f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:55.495938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dceb2-c9db-458a-b497-9a820102e2f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:55.497805 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2b0dceb2-c9db-458a-b497-9a820102e2f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:55.507131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c4d87665cb49e18aff95012dd33eb6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:55.520021 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c4d87665cb49e18aff95012dd33eb6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:55.528708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c4d87665cb49e18aff95012dd33eb6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:55.562754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dceb2-c9db-458a-b497-9a820102e2f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:55.564274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dceb2-c9db-458a-b497-9a820102e2f4 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:55.574151 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1365/2703] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 09:14:55.589109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c4d87665cb49e18aff95012dd33eb6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:55.589732 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5885d93d-67e0-4f26-b4d1-8ac97a770a01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c4d87665cb49e18aff95012dd33eb6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:55.597595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1339/2704] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:55 2022] DELETE /identity/v3/users/53c4d87665cb49e18aff95012dd33eb6 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:56.212192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dceb2-c9db-458a-b497-9a820102e2f4 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:56.213649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2b0dceb2-c9db-458a-b497-9a820102e2f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:56.227652 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dceb2-c9db-458a-b497-9a820102e2f4 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:56.290558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:56.293112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:56.293112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:56.293112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] PATH_INFO: `/v3/projects/c374b7c6f49f460bbe03e572d503c77a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:56.293418 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c374b7c6f49f460bbe03e572d503c77a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:56.295257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:56.582716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c374b7c6f49f460bbe03e572d503c77a', 'request_id': 'req-3c791eed-c770-477a-bb07-5b1beb73cf2c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:56.590382 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c374b7c6f49f460bbe03e572d503c77a', 'request_id': 'req-3c791eed-c770-477a-bb07-5b1beb73cf2c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:56.603073 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c374b7c6f49f460bbe03e572d503c77a', 'request_id': 'req-3c791eed-c770-477a-bb07-5b1beb73cf2c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:56.697670 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] The token cache is being invalidate because project c374b7c6f49f460bbe03e572d503c77a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:56.697953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:56.698595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3c791eed-c770-477a-bb07-5b1beb73cf2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:56.700146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1366/2705] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:56 2022] DELETE /identity/v3/projects/c374b7c6f49f460bbe03e572d503c77a => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:56.705008 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898a04-629b-4136-b447-a3b02d9430cf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:56.705490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898a04-629b-4136-b447-a3b02d9430cf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:56.705626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898a04-629b-4136-b447-a3b02d9430cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:56.724033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d6898a04-629b-4136-b447-a3b02d9430cf None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:56.725286 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d6898a04-629b-4136-b447-a3b02d9430cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:56.786607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1340/2706] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:14:56.795156 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cdfda-13e2-42cc-b247-d2eb954ac376 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:56.795483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cdfda-13e2-42cc-b247-d2eb954ac376 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:56.795787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cdfda-13e2-42cc-b247-d2eb954ac376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:56.803719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:56.815215 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-785cdfda-13e2-42cc-b247-d2eb954ac376 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:56.816520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-785cdfda-13e2-42cc-b247-d2eb954ac376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:56.825489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1367/2707] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:56.829744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6898a04-629b-4136-b447-a3b02d9430cf None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:56.840051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6898a04-629b-4136-b447-a3b02d9430cf None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:56.844637 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc54cce-f5fe-49ea-b71c-6c86f59ca84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:56.845294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc54cce-f5fe-49ea-b71c-6c86f59ca84f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:56.845569 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc54cce-f5fe-49ea-b71c-6c86f59ca84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:56.845872 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc54cce-f5fe-49ea-b71c-6c86f59ca84f admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:56.846628 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc54cce-f5fe-49ea-b71c-6c86f59ca84f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:14:56.846947 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc54cce-f5fe-49ea-b71c-6c86f59ca84f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:56.848530 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc54cce-f5fe-49ea-b71c-6c86f59ca84f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:56.850242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:56.861201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1341/2708] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:14:56.865521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cdfda-13e2-42cc-b247-d2eb954ac376 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:56.874562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cdfda-13e2-42cc-b247-d2eb954ac376 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:56.879524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64574-71c0-42b2-a5aa-c4babbbee980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:56.880236 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64574-71c0-42b2-a5aa-c4babbbee980 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:56.880523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64574-71c0-42b2-a5aa-c4babbbee980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:56.880799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64574-71c0-42b2-a5aa-c4babbbee980 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:56.882719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b64574-71c0-42b2-a5aa-c4babbbee980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:56.885493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b64574-71c0-42b2-a5aa-c4babbbee980 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:56.974177 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1368/2709] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:14:56 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 09:14:56.978445 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc54cce-f5fe-49ea-b71c-6c86f59ca84f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:56.988208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc54cce-f5fe-49ea-b71c-6c86f59ca84f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:57.047217 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32440847-f334-4c73-a391-8ce3588bafd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:57.047958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32440847-f334-4c73-a391-8ce3588bafd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:57.048240 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32440847-f334-4c73-a391-8ce3588bafd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:57.048513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32440847-f334-4c73-a391-8ce3588bafd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:57.049215 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32440847-f334-4c73-a391-8ce3588bafd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:57.051251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32440847-f334-4c73-a391-8ce3588bafd4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:57.112555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1342/2710] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:14:56 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:57.113783 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64574-71c0-42b2-a5aa-c4babbbee980 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:57.125169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64574-71c0-42b2-a5aa-c4babbbee980 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:57.129899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c49a3-1483-425d-a2a7-2bbaa1873678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:57.130590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c49a3-1483-425d-a2a7-2bbaa1873678 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:57.130870 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c49a3-1483-425d-a2a7-2bbaa1873678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:57.131537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c49a3-1483-425d-a2a7-2bbaa1873678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:57.132265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c49a3-1483-425d-a2a7-2bbaa1873678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:57.134791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c49a3-1483-425d-a2a7-2bbaa1873678 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:57.145598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1369/2711] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:57.150538 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32440847-f334-4c73-a391-8ce3588bafd4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:57.160497 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32440847-f334-4c73-a391-8ce3588bafd4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:57.165292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df9eff-57f6-4253-af69-6ffc1a6efeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:57.165952 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df9eff-57f6-4253-af69-6ffc1a6efeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:57.166264 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df9eff-57f6-4253-af69-6ffc1a6efeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:57.166536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df9eff-57f6-4253-af69-6ffc1a6efeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:57.167285 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df9eff-57f6-4253-af69-6ffc1a6efeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:57.168859 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df9eff-57f6-4253-af69-6ffc1a6efeb3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:57.178942 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1343/2712] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:57.184147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c49a3-1483-425d-a2a7-2bbaa1873678 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:57.194493 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c49a3-1483-425d-a2a7-2bbaa1873678 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:57.199944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c077af-38bb-4a32-a971-95b721052f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:57.200825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c077af-38bb-4a32-a971-95b721052f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:57.201106 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c077af-38bb-4a32-a971-95b721052f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:57.201376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c077af-38bb-4a32-a971-95b721052f62 admin admin] PATH_INFO: `/v3/projects/71887f4b66594c91af16e9ee68e99453/users/8c7a96f013e6477aa1e275aedcc4a25e/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:57.216390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c077af-38bb-4a32-a971-95b721052f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71887f4b66594c91af16e9ee68e99453, user_id=8c7a96f013e6477aa1e275aedcc4a25e, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:57.219059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c077af-38bb-4a32-a971-95b721052f62 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:57.269479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1370/2713] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:57 2022] PUT /identity/v3/projects/71887f4b66594c91af16e9ee68e99453/users/8c7a96f013e6477aa1e275aedcc4a25e/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:14:57.280074 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6dc3b-94a7-4d64-b05a-0e25431a73b2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:57.281816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6dc3b-94a7-4d64-b05a-0e25431a73b2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:57.281816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6dc3b-94a7-4d64-b05a-0e25431a73b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:57.308094 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6bc6dc3b-94a7-4d64-b05a-0e25431a73b2 None None] MFA Rules not processed for user `8c7a96f013e6477aa1e275aedcc4a25e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:57.309320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6bc6dc3b-94a7-4d64-b05a-0e25431a73b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:57.376567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1344/2714] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:57.381464 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c077af-38bb-4a32-a971-95b721052f62 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:57.453235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c077af-38bb-4a32-a971-95b721052f62 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:57.457922 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d236bb-337f-4d0f-9be0-6626b42ab18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:57.464260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d236bb-337f-4d0f-9be0-6626b42ab18f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:57.464260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d236bb-337f-4d0f-9be0-6626b42ab18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:57.464260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d236bb-337f-4d0f-9be0-6626b42ab18f admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:57.464260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d236bb-337f-4d0f-9be0-6626b42ab18f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:57.464260 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d236bb-337f-4d0f-9be0-6626b42ab18f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:57.623500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1371/2715] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:14:57 2022] POST /identity/v3/projects => generated 387 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:57.624665 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6dc3b-94a7-4d64-b05a-0e25431a73b2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:57.635894 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6dc3b-94a7-4d64-b05a-0e25431a73b2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:57.701411 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff360d9-74d5-4044-a45e-731e768a9bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:57.702169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff360d9-74d5-4044-a45e-731e768a9bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:57.702547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff360d9-74d5-4044-a45e-731e768a9bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:57.702883 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff360d9-74d5-4044-a45e-731e768a9bdc admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:57.703642 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff360d9-74d5-4044-a45e-731e768a9bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:57.705635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff360d9-74d5-4044-a45e-731e768a9bdc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:57.713740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:57.835092 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1345/2716] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:14:57 2022] POST /identity/v3/users => generated 300 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:14:57.839858 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d236bb-337f-4d0f-9be0-6626b42ab18f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:57.851527 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d236bb-337f-4d0f-9be0-6626b42ab18f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:57.861294 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992b72-b212-417f-88e0-94ca8472bcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:57.866025 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992b72-b212-417f-88e0-94ca8472bcff admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:57.866353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992b72-b212-417f-88e0-94ca8472bcff admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:57.866633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992b72-b212-417f-88e0-94ca8472bcff admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:57.867378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4992b72-b212-417f-88e0-94ca8472bcff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:57.869431 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4992b72-b212-417f-88e0-94ca8472bcff admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:57.881623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1372/2717] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:57.886400 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff360d9-74d5-4044-a45e-731e768a9bdc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:57.896824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff360d9-74d5-4044-a45e-731e768a9bdc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:57.902683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae872117-466b-4e10-9f57-90f146b03834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:57.903251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872117-466b-4e10-9f57-90f146b03834 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:57.903532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872117-466b-4e10-9f57-90f146b03834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:57.903805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872117-466b-4e10-9f57-90f146b03834 admin admin] PATH_INFO: `/v3/projects/f0e028d821c942e584f7ed5c2cbee06b/users/ab01c67149684a9e843b1f2ce1559751/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:57.925646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae872117-466b-4e10-9f57-90f146b03834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e028d821c942e584f7ed5c2cbee06b, user_id=ab01c67149684a9e843b1f2ce1559751, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:57.928403 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae872117-466b-4e10-9f57-90f146b03834 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:58.012960 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1346/2718] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:57 2022] PUT /identity/v3/projects/f0e028d821c942e584f7ed5c2cbee06b/users/ab01c67149684a9e843b1f2ce1559751/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:14:58.017016 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992b72-b212-417f-88e0-94ca8472bcff admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:58.027537 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992b72-b212-417f-88e0-94ca8472bcff admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:58.092634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1134e-9c26-4e77-8f60-584820530072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:58.093613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1134e-9c26-4e77-8f60-584820530072 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:58.094134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1134e-9c26-4e77-8f60-584820530072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:58.094587 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1134e-9c26-4e77-8f60-584820530072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:58.096197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1134e-9c26-4e77-8f60-584820530072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:58.097819 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1134e-9c26-4e77-8f60-584820530072 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:58.115240 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1373/2719] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:14:58.120049 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae872117-466b-4e10-9f57-90f146b03834 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:58.131747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae872117-466b-4e10-9f57-90f146b03834 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:58.138089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bf2c3-47da-403e-8e4f-4b1a346886af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:58.139117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bf2c3-47da-403e-8e4f-4b1a346886af admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:58.139500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bf2c3-47da-403e-8e4f-4b1a346886af admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:58.139874 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bf2c3-47da-403e-8e4f-4b1a346886af admin admin] PATH_INFO: `/v3/projects/f0e028d821c942e584f7ed5c2cbee06b/users/ab01c67149684a9e843b1f2ce1559751/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:58.142278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bf2c3-47da-403e-8e4f-4b1a346886af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e028d821c942e584f7ed5c2cbee06b, user_id=ab01c67149684a9e843b1f2ce1559751, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:58.145110 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bf2c3-47da-403e-8e4f-4b1a346886af admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:58.157464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1347/2720] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:14:58 2022] PUT /identity/v3/projects/f0e028d821c942e584f7ed5c2cbee06b/users/ab01c67149684a9e843b1f2ce1559751/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:58.240636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5942-db51-4d67-b489-8529b071058c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:58.240636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5942-db51-4d67-b489-8529b071058c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:58.240636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5942-db51-4d67-b489-8529b071058c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:58.253799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:58.269005 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fdca5942-db51-4d67-b489-8529b071058c None None] MFA Rules not processed for user `ab01c67149684a9e843b1f2ce1559751`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:58.269005 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fdca5942-db51-4d67-b489-8529b071058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:58.341498 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1374/2721] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:58.356224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bf2c3-47da-403e-8e4f-4b1a346886af admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:58.358197 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-425bf2c3-47da-403e-8e4f-4b1a346886af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:58.422194 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bf2c3-47da-403e-8e4f-4b1a346886af admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:58.430081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5234416-5022-40db-bb51-cc9738670110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:58.430081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5234416-5022-40db-bb51-cc9738670110 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:58.430081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5234416-5022-40db-bb51-cc9738670110 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:58.430081 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5234416-5022-40db-bb51-cc9738670110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:58.437843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5234416-5022-40db-bb51-cc9738670110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:58.439362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5234416-5022-40db-bb51-cc9738670110 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:58.452079 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1348/2722] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:58.589020 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca5942-db51-4d67-b489-8529b071058c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:58.651885 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca5942-db51-4d67-b489-8529b071058c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:58.656452 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:58.657311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:58.657589 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:58.657860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] PATH_INFO: `/v3/users/8c7a96f013e6477aa1e275aedcc4a25e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:58.659061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c7a96f013e6477aa1e275aedcc4a25e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:58.660971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:58.819943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c7a96f013e6477aa1e275aedcc4a25e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:58.827592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c7a96f013e6477aa1e275aedcc4a25e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:58.897842 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c7a96f013e6477aa1e275aedcc4a25e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:58.898514 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c7a96f013e6477aa1e275aedcc4a25e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:58.899183 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c7a96f013e6477aa1e275aedcc4a25e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:58.916372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1375/2723] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:58 2022] DELETE /identity/v3/users/8c7a96f013e6477aa1e275aedcc4a25e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:58.920944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd53fa1-e889-4f56-aed5-92980b08ebf5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:58.921420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd53fa1-e889-4f56-aed5-92980b08ebf5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:58.921420 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd53fa1-e889-4f56-aed5-92980b08ebf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:58.939270 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7bd53fa1-e889-4f56-aed5-92980b08ebf5 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:14:58.940449 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7bd53fa1-e889-4f56-aed5-92980b08ebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:58.997272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1349/2724] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:14:59.006958 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:59.009066 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:59.027330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:59.067151 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32617b7a-bbc9-4e28-976d-1825ee374a0f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:59.070969 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389d8e2-93aa-4843-be0b-8debaedee7bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:59.071548 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389d8e2-93aa-4843-be0b-8debaedee7bf service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:59.071828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389d8e2-93aa-4843-be0b-8debaedee7bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:59.072125 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389d8e2-93aa-4843-be0b-8debaedee7bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:59.080979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389d8e2-93aa-4843-be0b-8debaedee7bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:59.082430 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389d8e2-93aa-4843-be0b-8debaedee7bf service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:59.092314 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1376/2725] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:14:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:14:59.487900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd53fa1-e889-4f56-aed5-92980b08ebf5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:59.489383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7bd53fa1-e889-4f56-aed5-92980b08ebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:14:59.501747 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd53fa1-e889-4f56-aed5-92980b08ebf5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:59.506829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:59.507785 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:59.508451 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:59.508872 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] PATH_INFO: `/v3/projects/71887f4b66594c91af16e9ee68e99453` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:59.509935 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=71887f4b66594c91af16e9ee68e99453)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:59.511948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:59.623225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71887f4b66594c91af16e9ee68e99453', 'request_id': 'req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:59.625162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:14:59.628910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71887f4b66594c91af16e9ee68e99453', 'request_id': 'req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:59.703463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71887f4b66594c91af16e9ee68e99453', 'request_id': 'req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:59.715880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] The token cache is being invalidate because project 71887f4b66594c91af16e9ee68e99453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:14:59.716310 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:59.716931 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:59.718481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1350/2726] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:14:59 2022] DELETE /identity/v3/projects/71887f4b66594c91af16e9ee68e99453 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:14:59.722406 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389d8e2-93aa-4843-be0b-8debaedee7bf service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:14:59.733484 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389d8e2-93aa-4843-be0b-8debaedee7bf service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:14:59.788762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:14:59.790797 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:14:59.791078 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:14:59.791352 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] PATH_INFO: `/v3/users/ab01c67149684a9e843b1f2ce1559751` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:14:59.792543 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab01c67149684a9e843b1f2ce1559751)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:14:59.794517 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:14:59.974379 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab01c67149684a9e843b1f2ce1559751'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:14:59.981518 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab01c67149684a9e843b1f2ce1559751'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:00.051793 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab01c67149684a9e843b1f2ce1559751'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:00.052407 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab01c67149684a9e843b1f2ce1559751'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:00.053084 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-e9849e41-5cd0-4f3d-8374-be75a69544cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab01c67149684a9e843b1f2ce1559751'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:00.071158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1377/2727] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:14:59 2022] DELETE /identity/v3/users/ab01c67149684a9e843b1f2ce1559751 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:15:00.448529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:00.452814 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:00.465938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c68237-7e3a-49e0-8f2f-a59d9fb3a25c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:00.523271 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:00.524183 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:00.524461 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:00.524737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] PATH_INFO: `/v3/projects/f0e028d821c942e584f7ed5c2cbee06b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:00.525917 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e028d821c942e584f7ed5c2cbee06b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:00.527923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:00.618152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e028d821c942e584f7ed5c2cbee06b', 'request_id': 'req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:00.623472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e028d821c942e584f7ed5c2cbee06b', 'request_id': 'req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:00.691562 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e028d821c942e584f7ed5c2cbee06b', 'request_id': 'req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:00.702701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] The token cache is being invalidate because project f0e028d821c942e584f7ed5c2cbee06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:15:00.702994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:00.703631 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ff8112a9-a4e3-4c76-8fd6-6730ed9ac4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:00.705172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1351/2728] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:15:00 2022] DELETE /identity/v3/projects/f0e028d821c942e584f7ed5c2cbee06b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:15:00.710241 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff85711-acff-4943-8b7b-9aac1c4518c2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:00.710531 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff85711-acff-4943-8b7b-9aac1c4518c2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:00.715205 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff85711-acff-4943-8b7b-9aac1c4518c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:00.729107 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fff85711-acff-4943-8b7b-9aac1c4518c2 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:00.730324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fff85711-acff-4943-8b7b-9aac1c4518c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:00.763495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:00.790313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1378/2729] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:00.797915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0858089-2296-4398-817b-1d3c69face5b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:00.798224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0858089-2296-4398-817b-1d3c69face5b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:00.798501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0858089-2296-4398-817b-1d3c69face5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:00.818257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c0858089-2296-4398-817b-1d3c69face5b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:00.819444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c0858089-2296-4398-817b-1d3c69face5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:00.827981 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1352/2730] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:00.832076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff85711-acff-4943-8b7b-9aac1c4518c2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:00.841998 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff85711-acff-4943-8b7b-9aac1c4518c2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:00.846546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cc1b8-ac21-41a7-890b-8235d8721b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:00.847189 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cc1b8-ac21-41a7-890b-8235d8721b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:00.847467 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cc1b8-ac21-41a7-890b-8235d8721b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:00.847741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cc1b8-ac21-41a7-890b-8235d8721b8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:00.848457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cc1b8-ac21-41a7-890b-8235d8721b8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:15:00.848761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cc1b8-ac21-41a7-890b-8235d8721b8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:00.850451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cc1b8-ac21-41a7-890b-8235d8721b8e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:00.863132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1379/2731] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:15:00.866936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0858089-2296-4398-817b-1d3c69face5b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:00.877045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0858089-2296-4398-817b-1d3c69face5b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:00.881871 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e9841-ee86-4a01-8831-dc598e0d050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:00.882528 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e9841-ee86-4a01-8831-dc598e0d050c admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:00.882805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e9841-ee86-4a01-8831-dc598e0d050c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:00.883532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e9841-ee86-4a01-8831-dc598e0d050c admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:00.884253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e9841-ee86-4a01-8831-dc598e0d050c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:00.886196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e9841-ee86-4a01-8831-dc598e0d050c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:01.002730 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1353/2732] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:15:00 2022] POST /identity/v3/projects => generated 361 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:15:01.007139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cc1b8-ac21-41a7-890b-8235d8721b8e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:01.017512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cc1b8-ac21-41a7-890b-8235d8721b8e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:01.073188 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89698619-a919-46ec-aa11-71b8a1906d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:01.073813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89698619-a919-46ec-aa11-71b8a1906d7d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:01.074128 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89698619-a919-46ec-aa11-71b8a1906d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:01.074410 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89698619-a919-46ec-aa11-71b8a1906d7d admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:01.075083 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89698619-a919-46ec-aa11-71b8a1906d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:01.077012 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89698619-a919-46ec-aa11-71b8a1906d7d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:01.149675 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1380/2733] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:15:01 2022] POST /identity/v3/users => generated 281 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:15:01.153740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e9841-ee86-4a01-8831-dc598e0d050c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:01.163995 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e9841-ee86-4a01-8831-dc598e0d050c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:01.169438 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0037ac0-123b-4fd6-aba7-126322546228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:01.170124 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0037ac0-123b-4fd6-aba7-126322546228 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:01.170407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0037ac0-123b-4fd6-aba7-126322546228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:01.170682 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0037ac0-123b-4fd6-aba7-126322546228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:01.171396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0037ac0-123b-4fd6-aba7-126322546228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:01.172975 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0037ac0-123b-4fd6-aba7-126322546228 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:01.183453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1354/2734] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:15:01.187390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89698619-a919-46ec-aa11-71b8a1906d7d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:01.197127 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89698619-a919-46ec-aa11-71b8a1906d7d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:01.201672 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0880f-5a9a-4e70-bdbc-46010bef88a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:01.202355 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0880f-5a9a-4e70-bdbc-46010bef88a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:01.202639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0880f-5a9a-4e70-bdbc-46010bef88a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:01.202916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0880f-5a9a-4e70-bdbc-46010bef88a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:01.203636 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0880f-5a9a-4e70-bdbc-46010bef88a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:01.205221 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0880f-5a9a-4e70-bdbc-46010bef88a3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:01.215423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1381/2735] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:15:01.219360 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0037ac0-123b-4fd6-aba7-126322546228 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:01.229595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0037ac0-123b-4fd6-aba7-126322546228 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:01.234164 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651e190-7d62-47f1-b5a7-f25f98fa8e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:01.235025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e190-7d62-47f1-b5a7-f25f98fa8e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:01.235302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e190-7d62-47f1-b5a7-f25f98fa8e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:01.235573 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e190-7d62-47f1-b5a7-f25f98fa8e64 admin admin] PATH_INFO: `/v3/projects/1baf05f97b944b1da3f79c41cfbfd308/users/236907c49ce84c68a74a6fb3d1bcc2f2/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:01.250529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651e190-7d62-47f1-b5a7-f25f98fa8e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1baf05f97b944b1da3f79c41cfbfd308, user_id=236907c49ce84c68a74a6fb3d1bcc2f2, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:01.253130 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651e190-7d62-47f1-b5a7-f25f98fa8e64 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:01.342936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1355/2736] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:15:01 2022] PUT /identity/v3/projects/1baf05f97b944b1da3f79c41cfbfd308/users/236907c49ce84c68a74a6fb3d1bcc2f2/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:15:01.353732 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0511cd0-4337-4e50-8ba7-488a79169e2f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:01.354023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0511cd0-4337-4e50-8ba7-488a79169e2f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:01.354378 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0511cd0-4337-4e50-8ba7-488a79169e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:01.380547 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c0511cd0-4337-4e50-8ba7-488a79169e2f None None] MFA Rules not processed for user `236907c49ce84c68a74a6fb3d1bcc2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:01.382092 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c0511cd0-4337-4e50-8ba7-488a79169e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:01.444918 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1382/2737] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:01.481406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651e190-7d62-47f1-b5a7-f25f98fa8e64 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:01.483196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8651e190-7d62-47f1-b5a7-f25f98fa8e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:01.493396 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:01.541080 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651e190-7d62-47f1-b5a7-f25f98fa8e64 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:01.544757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f752d9-8916-4c36-aafa-f260556fc548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:01.545406 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f752d9-8916-4c36-aafa-f260556fc548 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:01.545690 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f752d9-8916-4c36-aafa-f260556fc548 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:01.545967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f752d9-8916-4c36-aafa-f260556fc548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:01.554645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f752d9-8916-4c36-aafa-f260556fc548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:01.556219 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f752d9-8916-4c36-aafa-f260556fc548 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:01.565227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1356/2738] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:15:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:15:01.581502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0511cd0-4337-4e50-8ba7-488a79169e2f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:01.616614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:01.643854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0511cd0-4337-4e50-8ba7-488a79169e2f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:01.648366 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:01.650061 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:01.650331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:01.650613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] PATH_INFO: `/v3/users/236907c49ce84c68a74a6fb3d1bcc2f2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:01.652077 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=236907c49ce84c68a74a6fb3d1bcc2f2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:01.654169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:01.818123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236907c49ce84c68a74a6fb3d1bcc2f2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:01.825612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236907c49ce84c68a74a6fb3d1bcc2f2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:01.878826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236907c49ce84c68a74a6fb3d1bcc2f2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:01.879376 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236907c49ce84c68a74a6fb3d1bcc2f2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:01.879974 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236907c49ce84c68a74a6fb3d1bcc2f2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:01.898054 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1383/2739] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:15:01 2022] DELETE /identity/v3/users/236907c49ce84c68a74a6fb3d1bcc2f2 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:15:01.902571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e89e68-1039-4bd7-95a6-2b4b57a07b25 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:01.902861 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e89e68-1039-4bd7-95a6-2b4b57a07b25 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:01.903141 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e89e68-1039-4bd7-95a6-2b4b57a07b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:01.920514 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-01e89e68-1039-4bd7-95a6-2b4b57a07b25 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:01.921735 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-01e89e68-1039-4bd7-95a6-2b4b57a07b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:01.979605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1357/2740] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:01.989101 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:01.990546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:02.047813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2be842-d19c-48b8-aecf-d778fecc1c5d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:02.051453 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceac11f-b154-4f4a-ab57-942897407cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:02.052087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceac11f-b154-4f4a-ab57-942897407cdb service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:02.052369 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceac11f-b154-4f4a-ab57-942897407cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:02.052644 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceac11f-b154-4f4a-ab57-942897407cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:02.062387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceac11f-b154-4f4a-ab57-942897407cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:02.063804 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceac11f-b154-4f4a-ab57-942897407cdb service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:02.073230 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1384/2741] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:15:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:15:02.481210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e89e68-1039-4bd7-95a6-2b4b57a07b25 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:02.482844 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-01e89e68-1039-4bd7-95a6-2b4b57a07b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:02.495670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e89e68-1039-4bd7-95a6-2b4b57a07b25 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:02.500463 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:02.505793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:02.505793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:02.505793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] PATH_INFO: `/v3/projects/1baf05f97b944b1da3f79c41cfbfd308` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:02.505793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1baf05f97b944b1da3f79c41cfbfd308)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:02.507149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:02.623162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1baf05f97b944b1da3f79c41cfbfd308', 'request_id': 'req-0a38f91d-648c-4d80-9447-c445a441286d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:02.628673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1baf05f97b944b1da3f79c41cfbfd308', 'request_id': 'req-0a38f91d-648c-4d80-9447-c445a441286d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:02.701809 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1baf05f97b944b1da3f79c41cfbfd308', 'request_id': 'req-0a38f91d-648c-4d80-9447-c445a441286d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:02.713044 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] The token cache is being invalidate because project 1baf05f97b944b1da3f79c41cfbfd308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:15:02.713327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:02.713951 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0a38f91d-648c-4d80-9447-c445a441286d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:02.715626 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1358/2742] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:15:02 2022] DELETE /identity/v3/projects/1baf05f97b944b1da3f79c41cfbfd308 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:15:02.720721 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a99d6-5d83-41c9-9b0b-4bcf039f7f47 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:02.721027 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a99d6-5d83-41c9-9b0b-4bcf039f7f47 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:02.721294 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a99d6-5d83-41c9-9b0b-4bcf039f7f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:02.742940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-de8a99d6-5d83-41c9-9b0b-4bcf039f7f47 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:02.742940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-de8a99d6-5d83-41c9-9b0b-4bcf039f7f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:02.787985 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:02.799841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1385/2743] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:02.808720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9195607-b4a7-40da-819a-0476b826728b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:02.809107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9195607-b4a7-40da-819a-0476b826728b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:02.809493 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9195607-b4a7-40da-819a-0476b826728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:02.827729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d9195607-b4a7-40da-819a-0476b826728b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:02.828982 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d9195607-b4a7-40da-819a-0476b826728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:02.836949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1359/2744] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:15:02.841334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a99d6-5d83-41c9-9b0b-4bcf039f7f47 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:02.851019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a99d6-5d83-41c9-9b0b-4bcf039f7f47 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:02.856711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e641d-87e0-47f5-a01b-9b7206f8afba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:02.861093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e641d-87e0-47f5-a01b-9b7206f8afba admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:02.861093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e641d-87e0-47f5-a01b-9b7206f8afba admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:02.861093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e641d-87e0-47f5-a01b-9b7206f8afba admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:02.861093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e641d-87e0-47f5-a01b-9b7206f8afba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:15:02.861093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e641d-87e0-47f5-a01b-9b7206f8afba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:02.861093 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e641d-87e0-47f5-a01b-9b7206f8afba admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:02.873743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1386/2745] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:15:02.878019 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9195607-b4a7-40da-819a-0476b826728b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:02.887649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9195607-b4a7-40da-819a-0476b826728b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:02.892272 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebc567-fd0d-4609-9e74-66aacb2d686f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:02.892944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebc567-fd0d-4609-9e74-66aacb2d686f admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:02.893227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebc567-fd0d-4609-9e74-66aacb2d686f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:02.893503 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebc567-fd0d-4609-9e74-66aacb2d686f admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:02.894226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebc567-fd0d-4609-9e74-66aacb2d686f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:02.896101 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebc567-fd0d-4609-9e74-66aacb2d686f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:03.008372 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1360/2746] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:15:02 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:15:03.012162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e641d-87e0-47f5-a01b-9b7206f8afba admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:03.022181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e641d-87e0-47f5-a01b-9b7206f8afba admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:03.078385 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531e58c-249f-433f-92b3-b077f6063b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:03.079045 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531e58c-249f-433f-92b3-b077f6063b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:03.079329 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531e58c-249f-433f-92b3-b077f6063b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:03.079605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531e58c-249f-433f-92b3-b077f6063b26 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:03.080283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531e58c-249f-433f-92b3-b077f6063b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:03.082219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531e58c-249f-433f-92b3-b077f6063b26 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:03.184048 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1387/2747] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:15:03 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:15:03.188423 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebc567-fd0d-4609-9e74-66aacb2d686f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:03.195633 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:03.199643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebc567-fd0d-4609-9e74-66aacb2d686f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:03.204323 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f31fba-e579-46c1-9590-bfe616e497ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:03.204970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31fba-e579-46c1-9590-bfe616e497ed admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:03.205278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31fba-e579-46c1-9590-bfe616e497ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:03.205582 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31fba-e579-46c1-9590-bfe616e497ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:03.206462 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f31fba-e579-46c1-9590-bfe616e497ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:03.208041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f31fba-e579-46c1-9590-bfe616e497ed admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:03.218925 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1361/2748] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:15:03.222776 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531e58c-249f-433f-92b3-b077f6063b26 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:03.233767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531e58c-249f-433f-92b3-b077f6063b26 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:03.238436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91285433-0fea-4485-a464-3531ebd217e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:03.239087 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91285433-0fea-4485-a464-3531ebd217e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:03.239388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91285433-0fea-4485-a464-3531ebd217e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:03.239660 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91285433-0fea-4485-a464-3531ebd217e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:03.240412 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91285433-0fea-4485-a464-3531ebd217e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:03.242010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91285433-0fea-4485-a464-3531ebd217e7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:03.252034 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1388/2749] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:15:03.256384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f31fba-e579-46c1-9590-bfe616e497ed admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:03.266464 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f31fba-e579-46c1-9590-bfe616e497ed admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:03.271595 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc7692-bc99-441d-9aab-c570993e7c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:03.272460 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc7692-bc99-441d-9aab-c570993e7c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:03.272733 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc7692-bc99-441d-9aab-c570993e7c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:03.272998 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc7692-bc99-441d-9aab-c570993e7c2c admin admin] PATH_INFO: `/v3/projects/affa29bd30b9411682914e34e81fcd62/users/3777321bda9f49978541823b45423a4c/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:03.287944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc7692-bc99-441d-9aab-c570993e7c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=affa29bd30b9411682914e34e81fcd62, user_id=3777321bda9f49978541823b45423a4c, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:03.291313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc7692-bc99-441d-9aab-c570993e7c2c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:03.372356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1362/2750] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:15:03 2022] PUT /identity/v3/projects/affa29bd30b9411682914e34e81fcd62/users/3777321bda9f49978541823b45423a4c/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:15:03.382831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f73274-ac76-4a15-8014-03b898f3316c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:03.383152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f73274-ac76-4a15-8014-03b898f3316c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:03.383435 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f73274-ac76-4a15-8014-03b898f3316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:03.409795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-33f73274-ac76-4a15-8014-03b898f3316c None None] MFA Rules not processed for user `3777321bda9f49978541823b45423a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:03.411055 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-33f73274-ac76-4a15-8014-03b898f3316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:03.477310 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1389/2751] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:03.489307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc7692-bc99-441d-9aab-c570993e7c2c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:03.491018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bacc7692-bc99-441d-9aab-c570993e7c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:03.547970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc7692-bc99-441d-9aab-c570993e7c2c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:03.551658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd2163-0d58-4f58-8473-8e410c3eac37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:03.552290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd2163-0d58-4f58-8473-8e410c3eac37 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:03.552566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd2163-0d58-4f58-8473-8e410c3eac37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:03.552837 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd2163-0d58-4f58-8473-8e410c3eac37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:03.561232 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd2163-0d58-4f58-8473-8e410c3eac37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:03.562670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd2163-0d58-4f58-8473-8e410c3eac37 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:03.571345 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1363/2752] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:15:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:15:13.474500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:13.567432 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:27.430398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f73274-ac76-4a15-8014-03b898f3316c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:27.492371 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f73274-ac76-4a15-8014-03b898f3316c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:27.496952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:27.497813 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:27.498123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:27.498398 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] PATH_INFO: `/v3/users/3777321bda9f49978541823b45423a4c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:27.499560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] RBAC: Authorizing `identity:delete_user(user_id=3777321bda9f49978541823b45423a4c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:27.502158 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:27.716525 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3777321bda9f49978541823b45423a4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:27.723952 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3777321bda9f49978541823b45423a4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:27.777245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3777321bda9f49978541823b45423a4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:27.777898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3777321bda9f49978541823b45423a4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:27.778590 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3777321bda9f49978541823b45423a4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:27.797403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1390/2753] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:15:27 2022] DELETE /identity/v3/users/3777321bda9f49978541823b45423a4c => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:15:27.801798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37087fd-3123-42ee-af07-457aa04df157 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:27.802321 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37087fd-3123-42ee-af07-457aa04df157 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:27.802427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37087fd-3123-42ee-af07-457aa04df157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:27.819477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c37087fd-3123-42ee-af07-457aa04df157 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:27.820660 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c37087fd-3123-42ee-af07-457aa04df157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:27.878243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1364/2754] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:27.888192 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:27.889579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:27.946281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d3bcc-218d-4f88-8b2b-3b4c3073bbce admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:27.949880 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3c976-333d-4eda-87c1-f3acd5a3420a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:27.950549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3c976-333d-4eda-87c1-f3acd5a3420a service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:27.950833 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3c976-333d-4eda-87c1-f3acd5a3420a service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:27.951113 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3c976-333d-4eda-87c1-f3acd5a3420a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:27.960032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3c976-333d-4eda-87c1-f3acd5a3420a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:27.961456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3c976-333d-4eda-87c1-f3acd5a3420a service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:27.971403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1391/2755] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:15:28.350936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37087fd-3123-42ee-af07-457aa04df157 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:28.352352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c37087fd-3123-42ee-af07-457aa04df157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:28.364544 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37087fd-3123-42ee-af07-457aa04df157 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:28.369356 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:28.370661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:28.370661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:28.370880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] PATH_INFO: `/v3/projects/affa29bd30b9411682914e34e81fcd62` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:28.372042 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=affa29bd30b9411682914e34e81fcd62)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:28.374027 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:28.481169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'affa29bd30b9411682914e34e81fcd62', 'request_id': 'req-1920653e-c472-462b-a1bc-778b1e1d9c56'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:28.486724 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'affa29bd30b9411682914e34e81fcd62', 'request_id': 'req-1920653e-c472-462b-a1bc-778b1e1d9c56'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:28.577588 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'affa29bd30b9411682914e34e81fcd62', 'request_id': 'req-1920653e-c472-462b-a1bc-778b1e1d9c56'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:28.589682 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] The token cache is being invalidate because project affa29bd30b9411682914e34e81fcd62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:15:28.589963 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:28.590605 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1920653e-c472-462b-a1bc-778b1e1d9c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:28.592118 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1365/2756] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:15:28 2022] DELETE /identity/v3/projects/affa29bd30b9411682914e34e81fcd62 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:15:28.597552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94be4c-10e8-4509-9390-218d307f436c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:28.598079 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94be4c-10e8-4509-9390-218d307f436c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:28.598195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94be4c-10e8-4509-9390-218d307f436c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:28.615301 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-de94be4c-10e8-4509-9390-218d307f436c None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:28.616520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-de94be4c-10e8-4509-9390-218d307f436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:28.634120 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:28.676059 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1392/2757] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:28.684475 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55b015-9f1f-4169-ae4f-aed417ef53b3 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:28.684756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55b015-9f1f-4169-ae4f-aed417ef53b3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:28.685033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55b015-9f1f-4169-ae4f-aed417ef53b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:28.705291 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-cd55b015-9f1f-4169-ae4f-aed417ef53b3 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:28.706523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cd55b015-9f1f-4169-ae4f-aed417ef53b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:28.714455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1366/2758] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:28.718538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94be4c-10e8-4509-9390-218d307f436c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:28.728646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94be4c-10e8-4509-9390-218d307f436c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:28.733248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62f5ec-c1f1-42ef-956a-6150d8a45ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:28.733821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62f5ec-c1f1-42ef-956a-6150d8a45ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:28.734143 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62f5ec-c1f1-42ef-956a-6150d8a45ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:28.734546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62f5ec-c1f1-42ef-956a-6150d8a45ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:28.735278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62f5ec-c1f1-42ef-956a-6150d8a45ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:15:28.735583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62f5ec-c1f1-42ef-956a-6150d8a45ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:28.737224 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62f5ec-c1f1-42ef-956a-6150d8a45ed2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:28.748500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1393/2759] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:15:28.753164 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55b015-9f1f-4169-ae4f-aed417ef53b3 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:28.762590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55b015-9f1f-4169-ae4f-aed417ef53b3 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:28.767849 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62233e-7514-4c81-a46a-a88abfa1548d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:28.768491 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62233e-7514-4c81-a46a-a88abfa1548d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:28.768770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62233e-7514-4c81-a46a-a88abfa1548d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:28.769042 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62233e-7514-4c81-a46a-a88abfa1548d admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:28.769722 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62233e-7514-4c81-a46a-a88abfa1548d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:28.771635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62233e-7514-4c81-a46a-a88abfa1548d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:28.852654 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1367/2760] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:15:28 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:15:28.856661 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62f5ec-c1f1-42ef-956a-6150d8a45ed2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:28.867233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62f5ec-c1f1-42ef-956a-6150d8a45ed2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:28.924794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce46c8a-d71f-4b7a-97f4-0cc66877d390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:28.925454 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce46c8a-d71f-4b7a-97f4-0cc66877d390 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:28.925739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce46c8a-d71f-4b7a-97f4-0cc66877d390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:28.926017 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce46c8a-d71f-4b7a-97f4-0cc66877d390 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:28.926738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce46c8a-d71f-4b7a-97f4-0cc66877d390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:28.928656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce46c8a-d71f-4b7a-97f4-0cc66877d390 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:29.022408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1394/2761] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:15:28 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:15:29.026344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62233e-7514-4c81-a46a-a88abfa1548d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:29.036566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62233e-7514-4c81-a46a-a88abfa1548d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:29.041220 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba62a0-761f-4bdd-bd9a-c8b2815f4331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:29.041867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba62a0-761f-4bdd-bd9a-c8b2815f4331 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:29.042184 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba62a0-761f-4bdd-bd9a-c8b2815f4331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:29.042459 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba62a0-761f-4bdd-bd9a-c8b2815f4331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:29.043177 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba62a0-761f-4bdd-bd9a-c8b2815f4331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:29.044750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba62a0-761f-4bdd-bd9a-c8b2815f4331 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:29.055509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1368/2762] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:15:29.059313 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce46c8a-d71f-4b7a-97f4-0cc66877d390 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:29.070191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce46c8a-d71f-4b7a-97f4-0cc66877d390 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:29.074994 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44343bb3-c37c-4d65-b547-5fcb381dc90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:29.075643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343bb3-c37c-4d65-b547-5fcb381dc90c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:29.075919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343bb3-c37c-4d65-b547-5fcb381dc90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:29.076204 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343bb3-c37c-4d65-b547-5fcb381dc90c admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:29.076970 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44343bb3-c37c-4d65-b547-5fcb381dc90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:29.078663 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44343bb3-c37c-4d65-b547-5fcb381dc90c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:29.088583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1395/2763] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:15:29.092511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba62a0-761f-4bdd-bd9a-c8b2815f4331 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:29.102287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba62a0-761f-4bdd-bd9a-c8b2815f4331 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:29.106869 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19704f56-151a-415f-9f32-304c86f19650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:29.107734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19704f56-151a-415f-9f32-304c86f19650 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:29.108018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19704f56-151a-415f-9f32-304c86f19650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:29.108290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19704f56-151a-415f-9f32-304c86f19650 admin admin] PATH_INFO: `/v3/projects/2a067778c46b4eb286895cd9b178d8ad/users/dc323e1a54ee4c978f47679b850ef18c/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:29.123739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19704f56-151a-415f-9f32-304c86f19650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a067778c46b4eb286895cd9b178d8ad, user_id=dc323e1a54ee4c978f47679b850ef18c, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:29.127128 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19704f56-151a-415f-9f32-304c86f19650 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:29.209744 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1369/2764] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:15:29 2022] PUT /identity/v3/projects/2a067778c46b4eb286895cd9b178d8ad/users/dc323e1a54ee4c978f47679b850ef18c/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:15:29.221019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995c66d-a9c6-453f-9d73-11b37ccf6186 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:29.222701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995c66d-a9c6-453f-9d73-11b37ccf6186 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:29.222701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995c66d-a9c6-453f-9d73-11b37ccf6186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:29.251037 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e995c66d-a9c6-453f-9d73-11b37ccf6186 None None] MFA Rules not processed for user `dc323e1a54ee4c978f47679b850ef18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:29.252268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e995c66d-a9c6-453f-9d73-11b37ccf6186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:29.315099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1396/2765] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 09:15:29.325534 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19704f56-151a-415f-9f32-304c86f19650 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:29.327209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-19704f56-151a-415f-9f32-304c86f19650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:29.349484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:29.384756 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19704f56-151a-415f-9f32-304c86f19650 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:29.388333 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875b3bd-e014-416e-b3d5-5450c6cd7e5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:29.388973 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b3bd-e014-416e-b3d5-5450c6cd7e5b service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:29.389249 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b3bd-e014-416e-b3d5-5450c6cd7e5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:29.389522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b3bd-e014-416e-b3d5-5450c6cd7e5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:29.397928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875b3bd-e014-416e-b3d5-5450c6cd7e5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:29.399383 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875b3bd-e014-416e-b3d5-5450c6cd7e5b service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:29.408713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1370/2766] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:15:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:15:33.460734 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995c66d-a9c6-453f-9d73-11b37ccf6186 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:33.483154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:33.529195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995c66d-a9c6-453f-9d73-11b37ccf6186 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:33.534769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:33.535668 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:33.535944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:33.540550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] PATH_INFO: `/v3/users/dc323e1a54ee4c978f47679b850ef18c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:33.540550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc323e1a54ee4c978f47679b850ef18c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:33.540550 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:33.735046 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc323e1a54ee4c978f47679b850ef18c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:33.741979 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc323e1a54ee4c978f47679b850ef18c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:33.817592 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc323e1a54ee4c978f47679b850ef18c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:33.818244 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc323e1a54ee4c978f47679b850ef18c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:33.818937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc323e1a54ee4c978f47679b850ef18c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:33.837040 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1397/2767] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:15:33 2022] DELETE /identity/v3/users/dc323e1a54ee4c978f47679b850ef18c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:15:33.841700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23f9b-0b91-4eae-9139-1742b52fc89f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:33.842199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23f9b-0b91-4eae-9139-1742b52fc89f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:33.842362 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23f9b-0b91-4eae-9139-1742b52fc89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:33.861089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4fd23f9b-0b91-4eae-9139-1742b52fc89f None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:33.862341 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4fd23f9b-0b91-4eae-9139-1742b52fc89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:33.923172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1371/2768] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:33.932727 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:33.934222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:33.994854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674dd3c-9c9d-45a2-a9a6-a597ba5d924a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:33.998598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc676e-7862-4836-8ec5-c0eb6db618bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:33.999243 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc676e-7862-4836-8ec5-c0eb6db618bd service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:33.999524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc676e-7862-4836-8ec5-c0eb6db618bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:33.999972 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc676e-7862-4836-8ec5-c0eb6db618bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:34.009475 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc676e-7862-4836-8ec5-c0eb6db618bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:34.010971 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc676e-7862-4836-8ec5-c0eb6db618bd service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:34.021191 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1398/2769] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:15:34.523003 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd23f9b-0b91-4eae-9139-1742b52fc89f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:15:34.524549 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4fd23f9b-0b91-4eae-9139-1742b52fc89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:34.539681 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd23f9b-0b91-4eae-9139-1742b52fc89f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:15:34.546096 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:15:34.547495 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:34.547822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:34.548098 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] PATH_INFO: `/v3/projects/2a067778c46b4eb286895cd9b178d8ad` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:34.549397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a067778c46b4eb286895cd9b178d8ad)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:15:34.551346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:15:34.679675 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a067778c46b4eb286895cd9b178d8ad', 'request_id': 'req-41d08413-63ba-4ecf-98a3-87d02df963b9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:34.686824 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a067778c46b4eb286895cd9b178d8ad', 'request_id': 'req-41d08413-63ba-4ecf-98a3-87d02df963b9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:34.758547 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a067778c46b4eb286895cd9b178d8ad', 'request_id': 'req-41d08413-63ba-4ecf-98a3-87d02df963b9'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:34.771783 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] The token cache is being invalidate because project 2a067778c46b4eb286895cd9b178d8ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:15:34.780542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:34.780542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:15:34.780542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1372/2770] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:15:34 2022] DELETE /identity/v3/projects/2a067778c46b4eb286895cd9b178d8ad => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:15:44.016946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:44.041008 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8c3ba-7be2-4475-90a4-b2b84653e562 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:15:44.041323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8c3ba-7be2-4475-90a4-b2b84653e562 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:15:44.041598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8c3ba-7be2-4475-90a4-b2b84653e562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:15:44.059098 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8eb8c3ba-7be2-4475-90a4-b2b84653e562 None None] MFA Rules not processed for user `47d7d837aafd4db7bf46346db8b399a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:15:44.060342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8eb8c3ba-7be2-4475-90a4-b2b84653e562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:15:44.114112 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1399/2771] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:15:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:15:44.768633 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:15:54.124323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=25 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:16:19.633797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:19.635448 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:19.696576 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d08413-63ba-4ecf-98a3-87d02df963b9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:19.700178 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2182b3-1035-4a29-82c3-5b189b13e123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:19.700823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2182b3-1035-4a29-82c3-5b189b13e123 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:19.701103 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2182b3-1035-4a29-82c3-5b189b13e123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:19.701378 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2182b3-1035-4a29-82c3-5b189b13e123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:19.703204 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-be2182b3-1035-4a29-82c3-5b189b13e123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:19.775162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2182b3-1035-4a29-82c3-5b189b13e123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:19.776610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2182b3-1035-4a29-82c3-5b189b13e123 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:19.788949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1373/2772] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:16:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:16:20.968306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8c3ba-7be2-4475-90a4-b2b84653e562 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:20.969911 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8eb8c3ba-7be2-4475-90a4-b2b84653e562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:20.982701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8c3ba-7be2-4475-90a4-b2b84653e562 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:20.986335 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fdf5-38f3-4790-8f0d-5ab088915e17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:20.986977 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fdf5-38f3-4790-8f0d-5ab088915e17 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:20.987261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fdf5-38f3-4790-8f0d-5ab088915e17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:20.987544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fdf5-38f3-4790-8f0d-5ab088915e17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:20.989930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f855fdf5-38f3-4790-8f0d-5ab088915e17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:21.003651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855fdf5-38f3-4790-8f0d-5ab088915e17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:21.005336 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855fdf5-38f3-4790-8f0d-5ab088915e17 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:21.016205 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1400/2773] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:16:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:16:23.766770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2182b3-1035-4a29-82c3-5b189b13e123 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:23.775755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2182b3-1035-4a29-82c3-5b189b13e123 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:23.779374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514690f4-4c5f-4762-b2d1-4f06e6dd6cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:23.782431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514690f4-4c5f-4762-b2d1-4f06e6dd6cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:23.782431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514690f4-4c5f-4762-b2d1-4f06e6dd6cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:23.782431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514690f4-4c5f-4762-b2d1-4f06e6dd6cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:23.782431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-514690f4-4c5f-4762-b2d1-4f06e6dd6cb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:23.843843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514690f4-4c5f-4762-b2d1-4f06e6dd6cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:23.845255 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514690f4-4c5f-4762-b2d1-4f06e6dd6cb2 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:23.855225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1374/2774] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:16:31.011392 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:16:31.936633 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fdf5-38f3-4790-8f0d-5ab088915e17 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:31.938235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f855fdf5-38f3-4790-8f0d-5ab088915e17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:31.953562 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fdf5-38f3-4790-8f0d-5ab088915e17 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:31.959080 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:31.959941 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:31.960225 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:31.960491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] PATH_INFO: `/v3/users/0a9a75227440420fb3eab1685598f2dc` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:31.961701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a9a75227440420fb3eab1685598f2dc)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:31.963718 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:32.167512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a9a75227440420fb3eab1685598f2dc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:32.174577 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a9a75227440420fb3eab1685598f2dc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:32.268311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a9a75227440420fb3eab1685598f2dc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:32.268928 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a9a75227440420fb3eab1685598f2dc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:32.269523 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a9a75227440420fb3eab1685598f2dc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:32.287655 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1401/2775] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:16:31 2022] DELETE /identity/v3/users/0a9a75227440420fb3eab1685598f2dc => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:16:32.542324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514690f4-4c5f-4762-b2d1-4f06e6dd6cb2 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:32.553100 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514690f4-4c5f-4762-b2d1-4f06e6dd6cb2 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:32.587030 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:16:32.610023 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:32.610904 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:32.611187 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:32.611466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] PATH_INFO: `/v3/projects/6b5574138d6341268eae22b78af64765` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:32.612634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5574138d6341268eae22b78af64765)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:32.615059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:32.755920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5574138d6341268eae22b78af64765', 'request_id': 'req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:32.761612 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5574138d6341268eae22b78af64765', 'request_id': 'req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:32.833668 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5574138d6341268eae22b78af64765', 'request_id': 'req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:32.845214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] The token cache is being invalidate because project 6b5574138d6341268eae22b78af64765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:16:32.845500 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:32.846152 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:32.848248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1375/2776] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:16:32 2022] DELETE /identity/v3/projects/6b5574138d6341268eae22b78af64765 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:16:32.852211 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:32.853598 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:32.880930 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d4953-acbe-4e87-b7dd-90ab11c27175 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:32.939296 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:32.940167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:32.940524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:32.941091 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] PATH_INFO: `/v3/users/9b0ebcbf81b74188b0aca48ec6b122aa` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:32.942417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0ebcbf81b74188b0aca48ec6b122aa)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:32.944353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:33.127892 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0ebcbf81b74188b0aca48ec6b122aa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:33.135433 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0ebcbf81b74188b0aca48ec6b122aa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:33.228219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0ebcbf81b74188b0aca48ec6b122aa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:33.228901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0ebcbf81b74188b0aca48ec6b122aa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:33.229516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0ebcbf81b74188b0aca48ec6b122aa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:33.237883 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:16:33.247762 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1402/2777] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:16:32 2022] DELETE /identity/v3/users/9b0ebcbf81b74188b0aca48ec6b122aa => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:16:33.475734 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:33.477456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:33.490994 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cceab4d-f3fc-4543-b568-2b19ef44e7a1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:33.552239 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:33.553067 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:33.553344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:33.553615 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] PATH_INFO: `/v3/projects/b487644732884eaabfc43581d1950b6e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:33.554805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b487644732884eaabfc43581d1950b6e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:33.558577 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:33.719770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b487644732884eaabfc43581d1950b6e', 'request_id': 'req-8ff50660-5ece-4476-9025-5e65b5e92914'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:33.725404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b487644732884eaabfc43581d1950b6e', 'request_id': 'req-8ff50660-5ece-4476-9025-5e65b5e92914'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:33.793474 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b487644732884eaabfc43581d1950b6e', 'request_id': 'req-8ff50660-5ece-4476-9025-5e65b5e92914'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:33.795552 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:16:33.805912 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] The token cache is being invalidate because project b487644732884eaabfc43581d1950b6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:16:33.806199 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:33.806823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:33.808361 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1376/2778] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:16:33 2022] DELETE /identity/v3/projects/b487644732884eaabfc43581d1950b6e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:16:36.522342 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:36.524826 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:36.589905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c4497-92c1-4148-9810-915c19242379 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:36.645380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952b0e9-036a-4636-b281-d5ac83707e06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:36.646026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952b0e9-036a-4636-b281-d5ac83707e06 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:36.646334 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952b0e9-036a-4636-b281-d5ac83707e06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:36.646607 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952b0e9-036a-4636-b281-d5ac83707e06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:36.648938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8952b0e9-036a-4636-b281-d5ac83707e06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:36.658207 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952b0e9-036a-4636-b281-d5ac83707e06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:36.659624 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952b0e9-036a-4636-b281-d5ac83707e06 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:36.668843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1403/2779] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:16:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:16:42.518921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:42.520379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:42.532701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff50660-5ece-4476-9025-5e65b5e92914 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:42.536984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef550647-5a1f-479d-a3ce-2b6019d547bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:42.537623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef550647-5a1f-479d-a3ce-2b6019d547bb service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:42.537905 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef550647-5a1f-479d-a3ce-2b6019d547bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:42.538209 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef550647-5a1f-479d-a3ce-2b6019d547bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:42.539936 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ef550647-5a1f-479d-a3ce-2b6019d547bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:42.604636 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef550647-5a1f-479d-a3ce-2b6019d547bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:42.606090 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef550647-5a1f-479d-a3ce-2b6019d547bb service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:42.616419 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1377/2780] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:16:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:16:46.664929 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=49,keystone:DELETE=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:16:50.329397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952b0e9-036a-4636-b281-d5ac83707e06 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:50.330909 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8952b0e9-036a-4636-b281-d5ac83707e06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:50.350697 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952b0e9-036a-4636-b281-d5ac83707e06 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:50.355476 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:50.356350 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:50.356639 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:50.357798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] PATH_INFO: `/v3/users/6f101dab08684c2483f67a9357efeae8` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:50.358166 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f101dab08684c2483f67a9357efeae8)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:50.360118 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:50.579138 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f101dab08684c2483f67a9357efeae8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:50.587403 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f101dab08684c2483f67a9357efeae8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:50.671375 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f101dab08684c2483f67a9357efeae8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:50.672000 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f101dab08684c2483f67a9357efeae8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:50.672595 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f101dab08684c2483f67a9357efeae8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:50.691003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1404/2781] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:16:50 2022] DELETE /identity/v3/users/6f101dab08684c2483f67a9357efeae8 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:16:50.933063 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef550647-5a1f-479d-a3ce-2b6019d547bb service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:50.944535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef550647-5a1f-479d-a3ce-2b6019d547bb service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:51.006536 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:51.007329 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:51.007609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:51.007882 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] PATH_INFO: `/v3/projects/02051a1f629a48199bb1eb504c747294` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:51.009059 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02051a1f629a48199bb1eb504c747294)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:51.010989 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:51.126599 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02051a1f629a48199bb1eb504c747294', 'request_id': 'req-f7e52eb9-edbf-4567-a681-b18ce4b0e915'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:51.132956 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02051a1f629a48199bb1eb504c747294', 'request_id': 'req-f7e52eb9-edbf-4567-a681-b18ce4b0e915'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:51.216456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02051a1f629a48199bb1eb504c747294', 'request_id': 'req-f7e52eb9-edbf-4567-a681-b18ce4b0e915'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:51.229149 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] The token cache is being invalidate because project 02051a1f629a48199bb1eb504c747294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:16:51.229431 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:51.230109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:51.231716 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1378/2782] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:16:50 2022] DELETE /identity/v3/projects/02051a1f629a48199bb1eb504c747294 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:16:51.235642 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:51.237047 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:51.250387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84241d-220e-4227-9f06-aa9c87384a0b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:51.308441 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:51.309311 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:51.309593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:51.309868 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] PATH_INFO: `/v3/users/03e63690d4ee4947b647239d2ba98d2d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:51.311109 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03e63690d4ee4947b647239d2ba98d2d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:51.313181 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:51.488168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03e63690d4ee4947b647239d2ba98d2d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:51.495010 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03e63690d4ee4947b647239d2ba98d2d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:51.574518 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03e63690d4ee4947b647239d2ba98d2d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:51.575552 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03e63690d4ee4947b647239d2ba98d2d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:51.576585 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55d77272-3b35-4f8c-9376-b1048a54d781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03e63690d4ee4947b647239d2ba98d2d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:51.587173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:16:51.596949 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1405/2783] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:16:51 2022] DELETE /identity/v3/users/03e63690d4ee4947b647239d2ba98d2d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:16:51.833368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:51.834928 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:51.849287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e52eb9-edbf-4567-a681-b18ce4b0e915 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:51.872532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:16:51.912736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:51.913625 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:51.913909 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:51.914214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] PATH_INFO: `/v3/projects/b16f471ef00f4ee5a3afe56bfdba8ae9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:51.915409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16f471ef00f4ee5a3afe56bfdba8ae9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:51.917340 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:52.017698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16f471ef00f4ee5a3afe56bfdba8ae9', 'request_id': 'req-b203d5b3-c914-474e-9514-85208fb68fd3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:52.023374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16f471ef00f4ee5a3afe56bfdba8ae9', 'request_id': 'req-b203d5b3-c914-474e-9514-85208fb68fd3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:52.130914 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16f471ef00f4ee5a3afe56bfdba8ae9', 'request_id': 'req-b203d5b3-c914-474e-9514-85208fb68fd3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:52.142489 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] The token cache is being invalidate because project b16f471ef00f4ee5a3afe56bfdba8ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:16:52.142778 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:52.143490 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:16:52.145374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1379/2784] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:16:51 2022] DELETE /identity/v3/projects/b16f471ef00f4ee5a3afe56bfdba8ae9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:16:52.680820 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c7de5-7a8a-4e72-9078-bd745ec0a7b9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:52.681380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c7de5-7a8a-4e72-9078-bd745ec0a7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:52.681380 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c7de5-7a8a-4e72-9078-bd745ec0a7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:52.698205 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-991c7de5-7a8a-4e72-9078-bd745ec0a7b9 None None] MFA Rules not processed for user `47d7d837aafd4db7bf46346db8b399a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:16:52.699388 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-991c7de5-7a8a-4e72-9078-bd745ec0a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:52.754247 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1406/2785] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:16:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:16:53.058556 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:53.059970 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:53.116903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203d5b3-c914-474e-9514-85208fb68fd3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:53.120501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b62118-facd-4791-84a6-7b930a4cab72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:53.121146 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b62118-facd-4791-84a6-7b930a4cab72 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:53.121425 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b62118-facd-4791-84a6-7b930a4cab72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:53.121698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b62118-facd-4791-84a6-7b930a4cab72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:53.123484 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f8b62118-facd-4791-84a6-7b930a4cab72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:53.133673 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b62118-facd-4791-84a6-7b930a4cab72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:53.135120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b62118-facd-4791-84a6-7b930a4cab72 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:53.143285 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1380/2786] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:16:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:16:53.547337 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c7de5-7a8a-4e72-9078-bd745ec0a7b9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:53.548739 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-991c7de5-7a8a-4e72-9078-bd745ec0a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:53.621864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c7de5-7a8a-4e72-9078-bd745ec0a7b9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:53.626810 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d783d5-8988-4657-9c84-dd880d619c87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:53.627455 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d783d5-8988-4657-9c84-dd880d619c87 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:53.627738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d783d5-8988-4657-9c84-dd880d619c87 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:53.628019 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d783d5-8988-4657-9c84-dd880d619c87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:53.630462 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-97d783d5-8988-4657-9c84-dd880d619c87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:53.641576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d783d5-8988-4657-9c84-dd880d619c87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:53.643180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d783d5-8988-4657-9c84-dd880d619c87 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:53.652975 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1407/2787] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:16:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:16:57.127865 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b62118-facd-4791-84a6-7b930a4cab72 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:16:57.133901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f8b62118-facd-4791-84a6-7b930a4cab72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:57.189987 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:16:57.212832 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b62118-facd-4791-84a6-7b930a4cab72 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:16:57.219646 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a814a-4883-486a-aa29-beb27b99a5a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:16:57.220752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a814a-4883-486a-aa29-beb27b99a5a2 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:16:57.221035 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a814a-4883-486a-aa29-beb27b99a5a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:16:57.221313 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a814a-4883-486a-aa29-beb27b99a5a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:16:57.224358 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ac2a814a-4883-486a-aa29-beb27b99a5a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:16:57.239680 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a814a-4883-486a-aa29-beb27b99a5a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:16:57.242295 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a814a-4883-486a-aa29-beb27b99a5a2 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:16:57.254469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1381/2788] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:16:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:17:02.543217 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d783d5-8988-4657-9c84-dd880d619c87 service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:17:02.547285 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-97d783d5-8988-4657-9c84-dd880d619c87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:17:02.560662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d783d5-8988-4657-9c84-dd880d619c87 service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:17:02.566266 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237f505-f170-4004-a64f-f9f52e3916f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8af26e036f4e466ebb923771c1f5f502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:17:02.569916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237f505-f170-4004-a64f-f9f52e3916f3 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:17:02.569916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237f505-f170-4004-a64f-f9f52e3916f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:17:02.569916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237f505-f170-4004-a64f-f9f52e3916f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:17:02.569916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8237f505-f170-4004-a64f-f9f52e3916f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:17:02.581972 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237f505-f170-4004-a64f-f9f52e3916f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:17:02.583516 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237f505-f170-4004-a64f-f9f52e3916f3 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:17:02.595132 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1408/2789] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:17:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:17:07.353097 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=10 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:17:12.590391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=66,keystone:DELETE=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:00.571303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a814a-4883-486a-aa29-beb27b99a5a2 service nova] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:00.581477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a814a-4883-486a-aa29-beb27b99a5a2 service nova] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:00.585114 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6ea1f-33f1-40a2-9868-f2c1b8d251b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:00.585745 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6ea1f-33f1-40a2-9868-f2c1b8d251b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:00.586026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6ea1f-33f1-40a2-9868-f2c1b8d251b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:00.586376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6ea1f-33f1-40a2-9868-f2c1b8d251b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:00.588221 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4fc6ea1f-33f1-40a2-9868-f2c1b8d251b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:00.650695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6ea1f-33f1-40a2-9868-f2c1b8d251b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:00.652111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6ea1f-33f1-40a2-9868-f2c1b8d251b3 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:00.661892 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1382/2790] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:18:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:18:04.242521 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237f505-f170-4004-a64f-f9f52e3916f3 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:04.254764 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237f505-f170-4004-a64f-f9f52e3916f3 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:04.260278 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b957715-4f5d-4e03-8a2e-1f7a46fa6205 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:04.260735 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b957715-4f5d-4e03-8a2e-1f7a46fa6205 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:04.261015 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b957715-4f5d-4e03-8a2e-1f7a46fa6205 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:04.261293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b957715-4f5d-4e03-8a2e-1f7a46fa6205 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:04.263167 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0b957715-4f5d-4e03-8a2e-1f7a46fa6205 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:04.275714 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b957715-4f5d-4e03-8a2e-1f7a46fa6205 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:04.277292 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b957715-4f5d-4e03-8a2e-1f7a46fa6205 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:04.287649 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1409/2791] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:18:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:18:04.352890 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6ea1f-33f1-40a2-9868-f2c1b8d251b3 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:04.355829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4fc6ea1f-33f1-40a2-9868-f2c1b8d251b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:04.369161 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6ea1f-33f1-40a2-9868-f2c1b8d251b3 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:04.373831 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:04.374731 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:04.375039 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:04.375290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] PATH_INFO: `/v3/users/47d7d837aafd4db7bf46346db8b399a6` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:04.376468 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d7d837aafd4db7bf46346db8b399a6)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:04.378721 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:04.507457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d7d837aafd4db7bf46346db8b399a6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:04.515852 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d7d837aafd4db7bf46346db8b399a6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:04.529051 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d7d837aafd4db7bf46346db8b399a6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:04.601226 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d7d837aafd4db7bf46346db8b399a6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:04.601780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-60a334cb-f5d6-4ff5-8ab1-aa4d3c4a58be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d7d837aafd4db7bf46346db8b399a6'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:04.609120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1383/2792] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:18:04 2022] DELETE /identity/v3/users/47d7d837aafd4db7bf46346db8b399a6 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:04.857422 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b957715-4f5d-4e03-8a2e-1f7a46fa6205 service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:04.865715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b957715-4f5d-4e03-8a2e-1f7a46fa6205 service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:04.926671 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:04.927532 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:04.927815 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:04.928096 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] PATH_INFO: `/v3/projects/0486a8d3794f4b50a96d3477e14e344f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:04.929381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0486a8d3794f4b50a96d3477e14e344f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:04.931333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:05.044072 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0486a8d3794f4b50a96d3477e14e344f', 'request_id': 'req-d3f175a3-3ffb-4af1-845c-0031c69c26d8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:05.049495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0486a8d3794f4b50a96d3477e14e344f', 'request_id': 'req-d3f175a3-3ffb-4af1-845c-0031c69c26d8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:05.055502 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0486a8d3794f4b50a96d3477e14e344f', 'request_id': 'req-d3f175a3-3ffb-4af1-845c-0031c69c26d8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:05.134014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] The token cache is being invalidate because project 0486a8d3794f4b50a96d3477e14e344f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:18:05.134261 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:05.134869 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d3f175a3-3ffb-4af1-845c-0031c69c26d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:05.137474 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1410/2793] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:18:04 2022] DELETE /identity/v3/projects/0486a8d3794f4b50a96d3477e14e344f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:05.142529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7d06c-a4d6-4220-9f55-643680d58a97 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:05.144397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7d06c-a4d6-4220-9f55-643680d58a97 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:05.144397 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7d06c-a4d6-4220-9f55-643680d58a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:05.162720 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-08d7d06c-a4d6-4220-9f55-643680d58a97 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:05.163902 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-08d7d06c-a4d6-4220-9f55-643680d58a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:05.223479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1384/2794] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:05.232372 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b7476-1333-442a-9810-a6096cc30aff None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:05.232662 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b7476-1333-442a-9810-a6096cc30aff None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:05.232943 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b7476-1333-442a-9810-a6096cc30aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:05.251489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5a9b7476-1333-442a-9810-a6096cc30aff None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:05.252957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5a9b7476-1333-442a-9810-a6096cc30aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:05.261578 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1411/2795] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:05.265737 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7d06c-a4d6-4220-9f55-643680d58a97 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:05.275281 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7d06c-a4d6-4220-9f55-643680d58a97 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:05.279948 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66100f26-6cf6-4c48-8ebf-f6802c5acbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:05.280590 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66100f26-6cf6-4c48-8ebf-f6802c5acbdc admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:05.280867 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66100f26-6cf6-4c48-8ebf-f6802c5acbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:05.281140 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66100f26-6cf6-4c48-8ebf-f6802c5acbdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:05.281853 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66100f26-6cf6-4c48-8ebf-f6802c5acbdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:18:05.282193 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66100f26-6cf6-4c48-8ebf-f6802c5acbdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:05.283772 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66100f26-6cf6-4c48-8ebf-f6802c5acbdc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:05.285214 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:05.295703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1385/2796] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:18:05.302023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b7476-1333-442a-9810-a6096cc30aff None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:05.311849 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b7476-1333-442a-9810-a6096cc30aff None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:05.316674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a47f46-4cb8-4237-a092-71e882a46e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:05.317318 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a47f46-4cb8-4237-a092-71e882a46e43 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:05.317603 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a47f46-4cb8-4237-a092-71e882a46e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:05.317881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a47f46-4cb8-4237-a092-71e882a46e43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:05.318614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a47f46-4cb8-4237-a092-71e882a46e43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:05.320651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a47f46-4cb8-4237-a092-71e882a46e43 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:05.408996 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1412/2797] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:18:05 2022] POST /identity/v3/projects => generated 405 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:05.413196 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66100f26-6cf6-4c48-8ebf-f6802c5acbdc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:05.423307 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66100f26-6cf6-4c48-8ebf-f6802c5acbdc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:05.478429 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06b5dc-1c00-46fb-9aa8-c4bc52de0ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:05.479087 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06b5dc-1c00-46fb-9aa8-c4bc52de0ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:05.479440 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06b5dc-1c00-46fb-9aa8-c4bc52de0ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:05.479937 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06b5dc-1c00-46fb-9aa8-c4bc52de0ab5 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:05.480622 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06b5dc-1c00-46fb-9aa8-c4bc52de0ab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:05.482542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06b5dc-1c00-46fb-9aa8-c4bc52de0ab5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:05.558534 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1386/2798] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:18:05 2022] POST /identity/v3/users => generated 303 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:05.562689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a47f46-4cb8-4237-a092-71e882a46e43 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:05.573962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a47f46-4cb8-4237-a092-71e882a46e43 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:05.578867 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fb585-e915-4933-b379-194a7766b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:05.579529 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fb585-e915-4933-b379-194a7766b07b admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:05.584486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fb585-e915-4933-b379-194a7766b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:05.584486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fb585-e915-4933-b379-194a7766b07b admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:05.584486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fb585-e915-4933-b379-194a7766b07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:05.584486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fb585-e915-4933-b379-194a7766b07b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:05.593387 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1413/2799] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:05.597266 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06b5dc-1c00-46fb-9aa8-c4bc52de0ab5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:05.607974 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06b5dc-1c00-46fb-9aa8-c4bc52de0ab5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:05.613192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07b76e-c119-4ccb-8659-9d6c52601249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:05.613192 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07b76e-c119-4ccb-8659-9d6c52601249 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:05.613426 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07b76e-c119-4ccb-8659-9d6c52601249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:05.613657 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07b76e-c119-4ccb-8659-9d6c52601249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:05.614402 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07b76e-c119-4ccb-8659-9d6c52601249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:05.615961 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07b76e-c119-4ccb-8659-9d6c52601249 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:05.626779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1387/2800] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:05.636423 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fb585-e915-4933-b379-194a7766b07b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:05.641162 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fb585-e915-4933-b379-194a7766b07b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:05.645741 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe76fa-9e23-4146-b55e-050912b1c3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:05.646648 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe76fa-9e23-4146-b55e-050912b1c3b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:05.646933 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe76fa-9e23-4146-b55e-050912b1c3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:05.647210 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe76fa-9e23-4146-b55e-050912b1c3b0 admin admin] PATH_INFO: `/v3/projects/54479a5efd5e4d45a864e1a228dfbbdb/users/9307d34f07db4f8cab9fd497109d3158/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:05.665046 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe76fa-9e23-4146-b55e-050912b1c3b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54479a5efd5e4d45a864e1a228dfbbdb, user_id=9307d34f07db4f8cab9fd497109d3158, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:05.665046 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe76fa-9e23-4146-b55e-050912b1c3b0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:05.740588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1414/2801] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:18:05 2022] PUT /identity/v3/projects/54479a5efd5e4d45a864e1a228dfbbdb/users/9307d34f07db4f8cab9fd497109d3158/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:05.751212 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a18c8-19d9-416b-9990-1f2be670cfca None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:05.751502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a18c8-19d9-416b-9990-1f2be670cfca None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:05.751787 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a18c8-19d9-416b-9990-1f2be670cfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:05.778117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-884a18c8-19d9-416b-9990-1f2be670cfca None None] MFA Rules not processed for user `9307d34f07db4f8cab9fd497109d3158`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:05.779315 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-884a18c8-19d9-416b-9990-1f2be670cfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:05.841120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1388/2802] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:05.845494 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe76fa-9e23-4146-b55e-050912b1c3b0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:05.859357 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:05.908761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe76fa-9e23-4146-b55e-050912b1c3b0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:05.913468 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03309b-4cdd-4f92-9b33-b29e1987cc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:05.914201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03309b-4cdd-4f92-9b33-b29e1987cc1c admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:05.914483 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03309b-4cdd-4f92-9b33-b29e1987cc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:05.914763 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03309b-4cdd-4f92-9b33-b29e1987cc1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:05.915503 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03309b-4cdd-4f92-9b33-b29e1987cc1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:05.918519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03309b-4cdd-4f92-9b33-b29e1987cc1c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:06.045664 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1415/2803] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:18:05 2022] POST /identity/v3/projects => generated 405 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:06.049713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a18c8-19d9-416b-9990-1f2be670cfca None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:06.060287 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a18c8-19d9-416b-9990-1f2be670cfca None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:06.115844 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c7629-7e44-40e2-9ea9-59a3a8d1bd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:06.116498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c7629-7e44-40e2-9ea9-59a3a8d1bd0d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:06.116793 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c7629-7e44-40e2-9ea9-59a3a8d1bd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:06.117045 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c7629-7e44-40e2-9ea9-59a3a8d1bd0d admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:06.117746 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c7629-7e44-40e2-9ea9-59a3a8d1bd0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:06.119670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c7629-7e44-40e2-9ea9-59a3a8d1bd0d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:06.127651 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:06.224041 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1389/2804] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:18:06 2022] POST /identity/v3/users => generated 309 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:06.227882 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03309b-4cdd-4f92-9b33-b29e1987cc1c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:06.239089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03309b-4cdd-4f92-9b33-b29e1987cc1c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:06.243674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dfb87-1d14-4a5f-8d63-a74ade0b1ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:06.244320 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dfb87-1d14-4a5f-8d63-a74ade0b1ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:06.244601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dfb87-1d14-4a5f-8d63-a74ade0b1ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:06.244936 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dfb87-1d14-4a5f-8d63-a74ade0b1ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:06.245710 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0dfb87-1d14-4a5f-8d63-a74ade0b1ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:06.247400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0dfb87-1d14-4a5f-8d63-a74ade0b1ee9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:06.259257 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1416/2805] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:06.263185 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c7629-7e44-40e2-9ea9-59a3a8d1bd0d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:06.272976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c7629-7e44-40e2-9ea9-59a3a8d1bd0d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:06.278158 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e0545-5adb-47c1-8860-9dd96cfad4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:06.279018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e0545-5adb-47c1-8860-9dd96cfad4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:06.279293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e0545-5adb-47c1-8860-9dd96cfad4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:06.279586 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e0545-5adb-47c1-8860-9dd96cfad4c3 admin admin] PATH_INFO: `/v3/projects/ed46dc9e4ada4d99a6ac326865910f98/users/3866e5c302cc4fc0bb78bc7dea9436fe/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:06.294540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e0545-5adb-47c1-8860-9dd96cfad4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed46dc9e4ada4d99a6ac326865910f98, user_id=3866e5c302cc4fc0bb78bc7dea9436fe, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:06.297144 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e0545-5adb-47c1-8860-9dd96cfad4c3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:06.402529 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1390/2806] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:18:06 2022] PUT /identity/v3/projects/ed46dc9e4ada4d99a6ac326865910f98/users/3866e5c302cc4fc0bb78bc7dea9436fe/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:18:06.406615 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dfb87-1d14-4a5f-8d63-a74ade0b1ee9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:06.416666 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dfb87-1d14-4a5f-8d63-a74ade0b1ee9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:06.472491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec4f18-695b-4771-94bb-267ebc4100f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:06.473148 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec4f18-695b-4771-94bb-267ebc4100f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:06.473428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec4f18-695b-4771-94bb-267ebc4100f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:06.473701 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec4f18-695b-4771-94bb-267ebc4100f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:06.474496 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec4f18-695b-4771-94bb-267ebc4100f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:06.476032 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec4f18-695b-4771-94bb-267ebc4100f4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:06.485694 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1417/2807] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:06.489901 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e0545-5adb-47c1-8860-9dd96cfad4c3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:06.500634 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e0545-5adb-47c1-8860-9dd96cfad4c3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:06.505540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271adf23-ca24-4105-8b69-e444c2faf201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:06.506442 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271adf23-ca24-4105-8b69-e444c2faf201 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:06.506719 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271adf23-ca24-4105-8b69-e444c2faf201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:06.506985 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271adf23-ca24-4105-8b69-e444c2faf201 admin admin] PATH_INFO: `/v3/projects/ed46dc9e4ada4d99a6ac326865910f98/users/3866e5c302cc4fc0bb78bc7dea9436fe/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:06.508933 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271adf23-ca24-4105-8b69-e444c2faf201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed46dc9e4ada4d99a6ac326865910f98, user_id=3866e5c302cc4fc0bb78bc7dea9436fe, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:06.512472 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271adf23-ca24-4105-8b69-e444c2faf201 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:06.522498 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1391/2808] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:18:06 2022] PUT /identity/v3/projects/ed46dc9e4ada4d99a6ac326865910f98/users/3866e5c302cc4fc0bb78bc7dea9436fe/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:06.533227 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3cb90-6e6d-4cc1-8ea0-fc8c8b0e9f6a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:06.533514 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3cb90-6e6d-4cc1-8ea0-fc8c8b0e9f6a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:06.533791 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3cb90-6e6d-4cc1-8ea0-fc8c8b0e9f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:06.560193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bad3cb90-6e6d-4cc1-8ea0-fc8c8b0e9f6a None None] MFA Rules not processed for user `3866e5c302cc4fc0bb78bc7dea9436fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:06.561535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bad3cb90-6e6d-4cc1-8ea0-fc8c8b0e9f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:06.628690 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1418/2809] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:06 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:06.639926 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271adf23-ca24-4105-8b69-e444c2faf201 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:06.641689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-271adf23-ca24-4105-8b69-e444c2faf201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:06.698609 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271adf23-ca24-4105-8b69-e444c2faf201 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:06.702303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b27ee-7209-4f9f-9804-e17e79aab823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:06.702944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b27ee-7209-4f9f-9804-e17e79aab823 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:06.703228 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b27ee-7209-4f9f-9804-e17e79aab823 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:06.703510 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b27ee-7209-4f9f-9804-e17e79aab823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:06.712532 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b27ee-7209-4f9f-9804-e17e79aab823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:06.714659 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b27ee-7209-4f9f-9804-e17e79aab823 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:06.724251 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1392/2810] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:18:06 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:18:06.850625 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3cb90-6e6d-4cc1-8ea0-fc8c8b0e9f6a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:06.860193 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:06.912567 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3cb90-6e6d-4cc1-8ea0-fc8c8b0e9f6a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:06.917295 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:06.918220 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:06.918712 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:06.919353 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] PATH_INFO: `/v3/users/9307d34f07db4f8cab9fd497109d3158` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:06.920612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9307d34f07db4f8cab9fd497109d3158)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:06.922622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:07.106144 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9307d34f07db4f8cab9fd497109d3158'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:07.112925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9307d34f07db4f8cab9fd497109d3158'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:07.180485 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9307d34f07db4f8cab9fd497109d3158'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:07.181102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9307d34f07db4f8cab9fd497109d3158'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:07.181711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9307d34f07db4f8cab9fd497109d3158'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:07.200309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1419/2811] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:18:06 2022] DELETE /identity/v3/users/9307d34f07db4f8cab9fd497109d3158 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:07.204875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf2f04-a171-4aa5-9373-bafe95894777 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:07.205169 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf2f04-a171-4aa5-9373-bafe95894777 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:07.205447 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf2f04-a171-4aa5-9373-bafe95894777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:07.223150 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-cedf2f04-a171-4aa5-9373-bafe95894777 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:07.224379 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cedf2f04-a171-4aa5-9373-bafe95894777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:07.297180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1393/2812] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:07.302839 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:07.304262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:07.361738 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a5ffe-2a7d-4611-ac3a-4d088bf7d3c2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:07.365208 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54aa93-f844-47d0-ac8b-b704391694f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:07.365841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54aa93-f844-47d0-ac8b-b704391694f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:07.366159 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54aa93-f844-47d0-ac8b-b704391694f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:07.366436 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54aa93-f844-47d0-ac8b-b704391694f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:07.375937 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54aa93-f844-47d0-ac8b-b704391694f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:07.377618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54aa93-f844-47d0-ac8b-b704391694f1 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:07.386860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1420/2813] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:18:07.802384 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf2f04-a171-4aa5-9373-bafe95894777 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:07.806521 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cedf2f04-a171-4aa5-9373-bafe95894777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:07.824534 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf2f04-a171-4aa5-9373-bafe95894777 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:07.833988 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:07.834949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:07.835234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:07.835509 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] PATH_INFO: `/v3/projects/54479a5efd5e4d45a864e1a228dfbbdb` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:07.840797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=54479a5efd5e4d45a864e1a228dfbbdb)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:07.842765 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:07.948740 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54479a5efd5e4d45a864e1a228dfbbdb', 'request_id': 'req-afef0623-f3d8-4576-96e5-bf5bbc303a1c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:07.951043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:07.954845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54479a5efd5e4d45a864e1a228dfbbdb', 'request_id': 'req-afef0623-f3d8-4576-96e5-bf5bbc303a1c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:08.043347 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54479a5efd5e4d45a864e1a228dfbbdb', 'request_id': 'req-afef0623-f3d8-4576-96e5-bf5bbc303a1c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:08.055322 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] The token cache is being invalidate because project 54479a5efd5e4d45a864e1a228dfbbdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:18:08.055645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:08.056695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:08.058363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1394/2814] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:18:07 2022] DELETE /identity/v3/projects/54479a5efd5e4d45a864e1a228dfbbdb => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:08.062486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54aa93-f844-47d0-ac8b-b704391694f1 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:08.072535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54aa93-f844-47d0-ac8b-b704391694f1 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:08.102419 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:08.129584 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:08.130480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:08.130767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:08.131044 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] PATH_INFO: `/v3/users/3866e5c302cc4fc0bb78bc7dea9436fe` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:08.132229 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] RBAC: Authorizing `identity:delete_user(user_id=3866e5c302cc4fc0bb78bc7dea9436fe)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:08.134248 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:08.342456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3866e5c302cc4fc0bb78bc7dea9436fe'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:08.349465 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3866e5c302cc4fc0bb78bc7dea9436fe'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:08.431778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3866e5c302cc4fc0bb78bc7dea9436fe'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:08.432916 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3866e5c302cc4fc0bb78bc7dea9436fe'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:08.433609 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d20db73-1e1f-4e71-9479-c3241f4952af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3866e5c302cc4fc0bb78bc7dea9436fe'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:08.452194 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1421/2815] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:18:08 2022] DELETE /identity/v3/users/3866e5c302cc4fc0bb78bc7dea9436fe => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:08.893845 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:08.895424 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:08.908854 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef0623-f3d8-4576-96e5-bf5bbc303a1c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:08.966293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:08.967150 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:08.967427 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:08.967697 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] PATH_INFO: `/v3/projects/ed46dc9e4ada4d99a6ac326865910f98` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:08.968880 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed46dc9e4ada4d99a6ac326865910f98)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:08.970797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:09.082292 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed46dc9e4ada4d99a6ac326865910f98', 'request_id': 'req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:09.087649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed46dc9e4ada4d99a6ac326865910f98', 'request_id': 'req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:09.152093 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed46dc9e4ada4d99a6ac326865910f98', 'request_id': 'req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:09.163506 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] The token cache is being invalidate because project ed46dc9e4ada4d99a6ac326865910f98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:18:09.163725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:09.164344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9546a8d4-3eba-4cb1-b7eb-b25c738529ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:09.165863 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1395/2816] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:18:08 2022] DELETE /identity/v3/projects/ed46dc9e4ada4d99a6ac326865910f98 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:09.171281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf70ac-92aa-420c-9f23-bd8b25c6b43b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:09.171573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf70ac-92aa-420c-9f23-bd8b25c6b43b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:09.171878 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf70ac-92aa-420c-9f23-bd8b25c6b43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:09.190799 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-cccf70ac-92aa-420c-9f23-bd8b25c6b43b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:09.191992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cccf70ac-92aa-420c-9f23-bd8b25c6b43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:09.251141 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1422/2817] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:09.260458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ac99-db7e-4f48-ac71-8cf8be6ad644 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:09.260986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ac99-db7e-4f48-ac71-8cf8be6ad644 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:09.261324 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ac99-db7e-4f48-ac71-8cf8be6ad644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:09.280407 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a290ac99-db7e-4f48-ac71-8cf8be6ad644 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:09.281602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a290ac99-db7e-4f48-ac71-8cf8be6ad644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:09.289598 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1396/2818] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:09.294481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf70ac-92aa-420c-9f23-bd8b25c6b43b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:09.304306 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf70ac-92aa-420c-9f23-bd8b25c6b43b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:09.309519 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df17060-03a9-4384-a9a0-a65dd34a1a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:09.310198 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df17060-03a9-4384-a9a0-a65dd34a1a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:09.310480 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df17060-03a9-4384-a9a0-a65dd34a1a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:09.310752 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df17060-03a9-4384-a9a0-a65dd34a1a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:09.311501 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df17060-03a9-4384-a9a0-a65dd34a1a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:18:09.311798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df17060-03a9-4384-a9a0-a65dd34a1a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:09.313395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df17060-03a9-4384-a9a0-a65dd34a1a3f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:09.325289 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1423/2819] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:18:09.329441 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ac99-db7e-4f48-ac71-8cf8be6ad644 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:09.342230 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ac99-db7e-4f48-ac71-8cf8be6ad644 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:09.347806 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe434-43f7-4fa9-8395-a51c3002815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:09.348469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe434-43f7-4fa9-8395-a51c3002815e admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:09.348749 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe434-43f7-4fa9-8395-a51c3002815e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:09.349020 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe434-43f7-4fa9-8395-a51c3002815e admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:09.349703 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fe434-43f7-4fa9-8395-a51c3002815e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:09.351630 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fe434-43f7-4fa9-8395-a51c3002815e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:09.438469 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1397/2820] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:18:09 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:09.441508 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df17060-03a9-4384-a9a0-a65dd34a1a3f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:09.452604 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df17060-03a9-4384-a9a0-a65dd34a1a3f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:09.486960 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:09.510130 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f8401-7d32-4b0f-880e-487ad1526f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:09.511569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f8401-7d32-4b0f-880e-487ad1526f56 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:09.511896 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f8401-7d32-4b0f-880e-487ad1526f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:09.512168 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f8401-7d32-4b0f-880e-487ad1526f56 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:09.512859 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f8401-7d32-4b0f-880e-487ad1526f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:09.514808 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f8401-7d32-4b0f-880e-487ad1526f56 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:09.607309 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1424/2821] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:18:09 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:09.611236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe434-43f7-4fa9-8395-a51c3002815e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:09.621466 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe434-43f7-4fa9-8395-a51c3002815e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:09.626065 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecd63a-9b42-4dbc-bfd6-184422ef63f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:09.628539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecd63a-9b42-4dbc-bfd6-184422ef63f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:09.628539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecd63a-9b42-4dbc-bfd6-184422ef63f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:09.628539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecd63a-9b42-4dbc-bfd6-184422ef63f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:09.628539 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecd63a-9b42-4dbc-bfd6-184422ef63f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:09.629834 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecd63a-9b42-4dbc-bfd6-184422ef63f1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:09.640635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1398/2822] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 09:18:09.644614 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f8401-7d32-4b0f-880e-487ad1526f56 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:09.655232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f8401-7d32-4b0f-880e-487ad1526f56 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:09.662798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763daa29-f8e0-437e-8dd0-02bdb0be8a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:09.662798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763daa29-f8e0-437e-8dd0-02bdb0be8a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:09.662798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763daa29-f8e0-437e-8dd0-02bdb0be8a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:09.662798 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763daa29-f8e0-437e-8dd0-02bdb0be8a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:09.663228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763daa29-f8e0-437e-8dd0-02bdb0be8a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:09.663324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763daa29-f8e0-437e-8dd0-02bdb0be8a28 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:09.673823 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1425/2823] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:09.677966 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecd63a-9b42-4dbc-bfd6-184422ef63f1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:09.687962 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecd63a-9b42-4dbc-bfd6-184422ef63f1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:09.692501 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523a92b-56c3-4ffe-88e8-768a36be4ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:09.693370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523a92b-56c3-4ffe-88e8-768a36be4ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:09.693645 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523a92b-56c3-4ffe-88e8-768a36be4ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:09.693915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523a92b-56c3-4ffe-88e8-768a36be4ade admin admin] PATH_INFO: `/v3/projects/a3bd363aa5df4510bc00920eb719b996/users/0827e407a71844b1a07752c8617a06fa/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:09.708477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523a92b-56c3-4ffe-88e8-768a36be4ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3bd363aa5df4510bc00920eb719b996, user_id=0827e407a71844b1a07752c8617a06fa, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:09.711120 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523a92b-56c3-4ffe-88e8-768a36be4ade admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:09.774939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1399/2824] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:18:09 2022] PUT /identity/v3/projects/a3bd363aa5df4510bc00920eb719b996/users/0827e407a71844b1a07752c8617a06fa/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:18:09.785175 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9375ee-5dbb-4886-983e-87214b57c238 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:09.785512 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9375ee-5dbb-4886-983e-87214b57c238 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:09.785728 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9375ee-5dbb-4886-983e-87214b57c238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:09.813489 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ca9375ee-5dbb-4886-983e-87214b57c238 None None] MFA Rules not processed for user `0827e407a71844b1a07752c8617a06fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:09.814729 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ca9375ee-5dbb-4886-983e-87214b57c238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:09.876520 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1426/2825] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:09.881178 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523a92b-56c3-4ffe-88e8-768a36be4ade admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:09.890510 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:09.942367 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523a92b-56c3-4ffe-88e8-768a36be4ade admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:09.946996 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab57895-228e-4323-90fb-5ac4e822af02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:09.947639 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab57895-228e-4323-90fb-5ac4e822af02 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:09.948455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab57895-228e-4323-90fb-5ac4e822af02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:09.948789 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab57895-228e-4323-90fb-5ac4e822af02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:09.949584 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab57895-228e-4323-90fb-5ac4e822af02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:09.951487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab57895-228e-4323-90fb-5ac4e822af02 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:10.033444 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1400/2826] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:18:09 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:10.037426 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9375ee-5dbb-4886-983e-87214b57c238 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:10.048111 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9375ee-5dbb-4886-983e-87214b57c238 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:10.102989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69b3db-0e4a-4999-a357-513669b62d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:10.103650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b3db-0e4a-4999-a357-513669b62d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:10.103931 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b3db-0e4a-4999-a357-513669b62d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:10.104770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b3db-0e4a-4999-a357-513669b62d90 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:10.105456 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69b3db-0e4a-4999-a357-513669b62d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:10.107400 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69b3db-0e4a-4999-a357-513669b62d90 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:10.186767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1427/2827] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:18:10 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:10.190855 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab57895-228e-4323-90fb-5ac4e822af02 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:10.201363 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab57895-228e-4323-90fb-5ac4e822af02 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:10.205875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0bb8d-fe34-4b14-b704-b46612bff23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:10.206548 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0bb8d-fe34-4b14-b704-b46612bff23e admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:10.206835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0bb8d-fe34-4b14-b704-b46612bff23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:10.207104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0bb8d-fe34-4b14-b704-b46612bff23e admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:10.207823 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0bb8d-fe34-4b14-b704-b46612bff23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:10.209399 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0bb8d-fe34-4b14-b704-b46612bff23e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:10.219967 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1401/2828] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:10.223874 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69b3db-0e4a-4999-a357-513669b62d90 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:10.234321 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69b3db-0e4a-4999-a357-513669b62d90 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:10.238911 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0aba3-1cdd-4a42-b99e-857545f2c2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:10.239843 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0aba3-1cdd-4a42-b99e-857545f2c2aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:10.241451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0aba3-1cdd-4a42-b99e-857545f2c2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:10.241451 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0aba3-1cdd-4a42-b99e-857545f2c2aa admin admin] PATH_INFO: `/v3/projects/fca1170eb2e74778a6a3657e3e5c47af/users/a5fa0a8ed79b43ac82a8e1881a364326/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:10.255716 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0aba3-1cdd-4a42-b99e-857545f2c2aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca1170eb2e74778a6a3657e3e5c47af, user_id=a5fa0a8ed79b43ac82a8e1881a364326, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:10.258390 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0aba3-1cdd-4a42-b99e-857545f2c2aa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:10.323294 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1428/2829] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:18:10 2022] PUT /identity/v3/projects/fca1170eb2e74778a6a3657e3e5c47af/users/a5fa0a8ed79b43ac82a8e1881a364326/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:10.327409 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0bb8d-fe34-4b14-b704-b46612bff23e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:10.338274 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0bb8d-fe34-4b14-b704-b46612bff23e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:10.395296 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fbcd7-ab48-4605-8f3b-347cdaf1894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:10.395959 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fbcd7-ab48-4605-8f3b-347cdaf1894a admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:10.396242 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fbcd7-ab48-4605-8f3b-347cdaf1894a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:10.396519 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fbcd7-ab48-4605-8f3b-347cdaf1894a admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:10.397236 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fbcd7-ab48-4605-8f3b-347cdaf1894a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:10.398863 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fbcd7-ab48-4605-8f3b-347cdaf1894a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:10.408465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1402/2830] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:10.412602 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0aba3-1cdd-4a42-b99e-857545f2c2aa admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:10.423601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0aba3-1cdd-4a42-b99e-857545f2c2aa admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:10.428323 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb53899-9caf-436e-8448-a5eea6ab61ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:10.429197 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb53899-9caf-436e-8448-a5eea6ab61ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:10.429479 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb53899-9caf-436e-8448-a5eea6ab61ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:10.429758 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb53899-9caf-436e-8448-a5eea6ab61ca admin admin] PATH_INFO: `/v3/projects/fca1170eb2e74778a6a3657e3e5c47af/users/a5fa0a8ed79b43ac82a8e1881a364326/roles/2079875b2ff141a689d6eb49744f5c65` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:10.431844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb53899-9caf-436e-8448-a5eea6ab61ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca1170eb2e74778a6a3657e3e5c47af, user_id=a5fa0a8ed79b43ac82a8e1881a364326, role_id=2079875b2ff141a689d6eb49744f5c65)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:10.435401 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb53899-9caf-436e-8448-a5eea6ab61ca admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:10.440831 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:10.446349 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1429/2831] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:18:10 2022] PUT /identity/v3/projects/fca1170eb2e74778a6a3657e3e5c47af/users/a5fa0a8ed79b43ac82a8e1881a364326/roles/2079875b2ff141a689d6eb49744f5c65 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:10.462156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037f11c-96d0-41ac-91ac-22b359f7e5f4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:10.462156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037f11c-96d0-41ac-91ac-22b359f7e5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:10.462156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037f11c-96d0-41ac-91ac-22b359f7e5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:10.485211 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b037f11c-96d0-41ac-91ac-22b359f7e5f4 None None] MFA Rules not processed for user `a5fa0a8ed79b43ac82a8e1881a364326`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:10.486456 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b037f11c-96d0-41ac-91ac-22b359f7e5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:10.553084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1403/2832] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:10.563705 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb53899-9caf-436e-8448-a5eea6ab61ca admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:10.565411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7eb53899-9caf-436e-8448-a5eea6ab61ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:10.622333 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb53899-9caf-436e-8448-a5eea6ab61ca admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:10.625946 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45c3d6-e3dd-4580-ac90-22804062286d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:10.626622 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c3d6-e3dd-4580-ac90-22804062286d service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:10.626899 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c3d6-e3dd-4580-ac90-22804062286d service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:10.627169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c3d6-e3dd-4580-ac90-22804062286d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:10.636428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45c3d6-e3dd-4580-ac90-22804062286d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:10.640147 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45c3d6-e3dd-4580-ac90-22804062286d service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:10.647095 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1430/2833] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:18:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:18:11.169320 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037f11c-96d0-41ac-91ac-22b359f7e5f4 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:11.173566 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:11.232761 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037f11c-96d0-41ac-91ac-22b359f7e5f4 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:11.237278 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:11.238176 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:11.238457 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:11.238725 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] PATH_INFO: `/v3/users/0827e407a71844b1a07752c8617a06fa` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:11.239897 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0827e407a71844b1a07752c8617a06fa)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:11.241802 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:11.434596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0827e407a71844b1a07752c8617a06fa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:11.441290 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0827e407a71844b1a07752c8617a06fa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:11.453522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0827e407a71844b1a07752c8617a06fa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:11.523262 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0827e407a71844b1a07752c8617a06fa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:11.523870 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0827e407a71844b1a07752c8617a06fa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:11.530360 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1404/2834] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:18:11 2022] DELETE /identity/v3/users/0827e407a71844b1a07752c8617a06fa => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:11.534844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbd732-5ac5-4bc8-9c7e-20ee810f5bcb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:11.535324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbd732-5ac5-4bc8-9c7e-20ee810f5bcb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:11.535425 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbd732-5ac5-4bc8-9c7e-20ee810f5bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:11.553110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-29dbd732-5ac5-4bc8-9c7e-20ee810f5bcb None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:11.554312 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-29dbd732-5ac5-4bc8-9c7e-20ee810f5bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:11.613448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1431/2835] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:11.623763 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:11.626689 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:11.681033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642c856-961c-44f7-b7bd-485951585821 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:11.685208 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b33704-5dbe-4c46-a4bd-7c202d00d670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:11.685854 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b33704-5dbe-4c46-a4bd-7c202d00d670 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:11.686138 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b33704-5dbe-4c46-a4bd-7c202d00d670 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:11.686410 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b33704-5dbe-4c46-a4bd-7c202d00d670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:11.695231 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b33704-5dbe-4c46-a4bd-7c202d00d670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:11.696635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b33704-5dbe-4c46-a4bd-7c202d00d670 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:11.705886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1405/2836] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:18:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:18:12.154110 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbd732-5ac5-4bc8-9c7e-20ee810f5bcb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:12.155557 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-29dbd732-5ac5-4bc8-9c7e-20ee810f5bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:12.169825 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbd732-5ac5-4bc8-9c7e-20ee810f5bcb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:12.174574 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:12.176011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:12.176298 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:12.176573 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] PATH_INFO: `/v3/projects/a3bd363aa5df4510bc00920eb719b996` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:12.177953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3bd363aa5df4510bc00920eb719b996)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:12.179992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:12.297033 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3bd363aa5df4510bc00920eb719b996', 'request_id': 'req-ef63ae03-2c69-4cec-959b-75ef44d6c261'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:12.302583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3bd363aa5df4510bc00920eb719b996', 'request_id': 'req-ef63ae03-2c69-4cec-959b-75ef44d6c261'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:12.308651 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3bd363aa5df4510bc00920eb719b996', 'request_id': 'req-ef63ae03-2c69-4cec-959b-75ef44d6c261'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:12.391737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] The token cache is being invalidate because project a3bd363aa5df4510bc00920eb719b996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:18:12.392024 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:12.392719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:12.394375 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1432/2837] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:18:12 2022] DELETE /identity/v3/projects/a3bd363aa5df4510bc00920eb719b996 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:12.402567 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b33704-5dbe-4c46-a4bd-7c202d00d670 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:12.409042 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b33704-5dbe-4c46-a4bd-7c202d00d670 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:12.429125 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:12.466513 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:12.467374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:12.467649 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:12.467923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] PATH_INFO: `/v3/users/a5fa0a8ed79b43ac82a8e1881a364326` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:12.469200 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5fa0a8ed79b43ac82a8e1881a364326)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:12.471273 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:12.639940 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5fa0a8ed79b43ac82a8e1881a364326'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:12.646872 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5fa0a8ed79b43ac82a8e1881a364326'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:12.659018 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5fa0a8ed79b43ac82a8e1881a364326'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:12.743314 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5fa0a8ed79b43ac82a8e1881a364326'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:12.743939 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-42545b5a-7559-4a88-a00a-b45abbcffb4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5fa0a8ed79b43ac82a8e1881a364326'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:12.750481 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1406/2838] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:18:12 2022] DELETE /identity/v3/users/a5fa0a8ed79b43ac82a8e1881a364326 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:13.223524 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:13.244901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:13.244901 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63ae03-2c69-4cec-959b-75ef44d6c261 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:13.259014 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:13.295538 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:13.296397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:13.296687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:13.296962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] PATH_INFO: `/v3/projects/fca1170eb2e74778a6a3657e3e5c47af` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:13.298184 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca1170eb2e74778a6a3657e3e5c47af)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:13.300102 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:13.413769 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca1170eb2e74778a6a3657e3e5c47af', 'request_id': 'req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:13.419317 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca1170eb2e74778a6a3657e3e5c47af', 'request_id': 'req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:13.425408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca1170eb2e74778a6a3657e3e5c47af', 'request_id': 'req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:13.497873 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] The token cache is being invalidate because project fca1170eb2e74778a6a3657e3e5c47af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:18:13.498127 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:13.498794 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b49a5f92-45aa-43d3-8e2b-9c49f81aa7de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:13.500330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1433/2839] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:18:13 2022] DELETE /identity/v3/projects/fca1170eb2e74778a6a3657e3e5c47af => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:13.505426 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e520ad-4966-4774-9eaa-594bade06ca0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:13.505907 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e520ad-4966-4774-9eaa-594bade06ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:13.506005 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e520ad-4966-4774-9eaa-594bade06ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:13.524357 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c8e520ad-4966-4774-9eaa-594bade06ca0 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:13.525535 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c8e520ad-4966-4774-9eaa-594bade06ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:13.583516 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1407/2840] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:13.591245 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d62bf-b04d-49c8-a3c2-704ec6f2e6fb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:13.591535 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d62bf-b04d-49c8-a3c2-704ec6f2e6fb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:13.591816 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d62bf-b04d-49c8-a3c2-704ec6f2e6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:13.609915 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-288d62bf-b04d-49c8-a3c2-704ec6f2e6fb None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:13.611133 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-288d62bf-b04d-49c8-a3c2-704ec6f2e6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:13.618953 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1434/2841] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:13.624193 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e520ad-4966-4774-9eaa-594bade06ca0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:13.634025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e520ad-4966-4774-9eaa-594bade06ca0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:13.639193 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191e770-53b6-4fa2-8d75-25aa221d6960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:13.639836 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191e770-53b6-4fa2-8d75-25aa221d6960 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:13.640117 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191e770-53b6-4fa2-8d75-25aa221d6960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:13.640391 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191e770-53b6-4fa2-8d75-25aa221d6960 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:13.641111 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191e770-53b6-4fa2-8d75-25aa221d6960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:18:13.641413 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191e770-53b6-4fa2-8d75-25aa221d6960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:13.643026 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191e770-53b6-4fa2-8d75-25aa221d6960 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:13.654878 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1408/2842] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:18:13.659053 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d62bf-b04d-49c8-a3c2-704ec6f2e6fb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:13.668273 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d62bf-b04d-49c8-a3c2-704ec6f2e6fb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:13.672925 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1056a9-fe04-4629-ada1-08a026b4db79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:13.673564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1056a9-fe04-4629-ada1-08a026b4db79 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:13.673848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1056a9-fe04-4629-ada1-08a026b4db79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:13.674160 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1056a9-fe04-4629-ada1-08a026b4db79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:13.674855 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1056a9-fe04-4629-ada1-08a026b4db79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:13.676759 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1056a9-fe04-4629-ada1-08a026b4db79 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:13.747848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1435/2843] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:18:13 2022] POST /identity/v3/projects => generated 371 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:13.751923 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191e770-53b6-4fa2-8d75-25aa221d6960 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:13.763393 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191e770-53b6-4fa2-8d75-25aa221d6960 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:13.791627 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:13.818958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0e021-683d-4cd0-afdf-6c57445989b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:13.819643 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0e021-683d-4cd0-afdf-6c57445989b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:13.819920 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0e021-683d-4cd0-afdf-6c57445989b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:13.820188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0e021-683d-4cd0-afdf-6c57445989b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:13.820887 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0e021-683d-4cd0-afdf-6c57445989b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:13.823674 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0e021-683d-4cd0-afdf-6c57445989b2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:13.903225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1409/2844] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:18:13 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:13.907226 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1056a9-fe04-4629-ada1-08a026b4db79 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:13.918980 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1056a9-fe04-4629-ada1-08a026b4db79 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:13.923864 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cad93-d071-40c7-8a0d-235e06572a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:13.924513 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cad93-d071-40c7-8a0d-235e06572a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:13.924795 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cad93-d071-40c7-8a0d-235e06572a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:13.925067 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cad93-d071-40c7-8a0d-235e06572a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:13.925828 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cad93-d071-40c7-8a0d-235e06572a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:13.927448 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cad93-d071-40c7-8a0d-235e06572a9b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:13.938907 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1436/2845] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:13.942798 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0e021-683d-4cd0-afdf-6c57445989b2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:13.953040 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0e021-683d-4cd0-afdf-6c57445989b2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:13.957757 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1a0a5-06f7-4bda-be61-18a61aab2bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:13.958453 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1a0a5-06f7-4bda-be61-18a61aab2bce admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:13.958736 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1a0a5-06f7-4bda-be61-18a61aab2bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:13.959011 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1a0a5-06f7-4bda-be61-18a61aab2bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:13.959728 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1a0a5-06f7-4bda-be61-18a61aab2bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:13.961303 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1a0a5-06f7-4bda-be61-18a61aab2bce admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:13.971350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1410/2846] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:13.975268 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cad93-d071-40c7-8a0d-235e06572a9b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:13.985588 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cad93-d071-40c7-8a0d-235e06572a9b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:13.990381 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1101f-1117-413b-b88d-0172ec18ccdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:13.991255 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1101f-1117-413b-b88d-0172ec18ccdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:13.991541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1101f-1117-413b-b88d-0172ec18ccdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:13.991841 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1101f-1117-413b-b88d-0172ec18ccdd admin admin] PATH_INFO: `/v3/projects/2bf14af935074201ac07949b467b3c99/users/140df9c609ae4ba4add18165ef330c49/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:14.008233 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1101f-1117-413b-b88d-0172ec18ccdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf14af935074201ac07949b467b3c99, user_id=140df9c609ae4ba4add18165ef330c49, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:14.010989 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1101f-1117-413b-b88d-0172ec18ccdd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:14.083023 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1437/2847] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:18:13 2022] PUT /identity/v3/projects/2bf14af935074201ac07949b467b3c99/users/140df9c609ae4ba4add18165ef330c49/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:18:14.093182 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e711330-4d11-41ce-b0c1-f4ec2d6dff84 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:14.093477 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e711330-4d11-41ce-b0c1-f4ec2d6dff84 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:14.093752 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e711330-4d11-41ce-b0c1-f4ec2d6dff84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:14.120486 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-2e711330-4d11-41ce-b0c1-f4ec2d6dff84 None None] MFA Rules not processed for user `140df9c609ae4ba4add18165ef330c49`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:14.121698 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2e711330-4d11-41ce-b0c1-f4ec2d6dff84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:14.184055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1411/2848] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:14.195777 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1101f-1117-413b-b88d-0172ec18ccdd admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:14.197442 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cfb1101f-1117-413b-b88d-0172ec18ccdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:14.255272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1101f-1117-413b-b88d-0172ec18ccdd admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:14.259157 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a4f7c-3b0d-4949-8bdc-54e8111f835d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:14.259768 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a4f7c-3b0d-4949-8bdc-54e8111f835d service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:14.260049 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a4f7c-3b0d-4949-8bdc-54e8111f835d service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:14.260325 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a4f7c-3b0d-4949-8bdc-54e8111f835d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:14.265541 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:14.268924 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a4f7c-3b0d-4949-8bdc-54e8111f835d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:14.270477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a4f7c-3b0d-4949-8bdc-54e8111f835d service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:14.280330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1438/2849] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:18:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:18:14.299958 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e711330-4d11-41ce-b0c1-f4ec2d6dff84 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:14.363610 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e711330-4d11-41ce-b0c1-f4ec2d6dff84 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:14.368162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:14.369022 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:14.369301 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:14.370168 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] PATH_INFO: `/v3/users/140df9c609ae4ba4add18165ef330c49` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:14.371635 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] RBAC: Authorizing `identity:delete_user(user_id=140df9c609ae4ba4add18165ef330c49)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:14.373542 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:14.567213 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:14.571910 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '140df9c609ae4ba4add18165ef330c49'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:14.579145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '140df9c609ae4ba4add18165ef330c49'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:14.590479 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '140df9c609ae4ba4add18165ef330c49'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:14.694954 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '140df9c609ae4ba4add18165ef330c49'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:14.695602 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '140df9c609ae4ba4add18165ef330c49'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:14.702465 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1412/2850] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:18:14 2022] DELETE /identity/v3/users/140df9c609ae4ba4add18165ef330c49 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:14.706992 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fd9c8-fae9-474c-a984-f0e815c379bd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:14.707281 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fd9c8-fae9-474c-a984-f0e815c379bd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:14.707557 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fd9c8-fae9-474c-a984-f0e815c379bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:14.724787 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8c8fd9c8-fae9-474c-a984-f0e815c379bd None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:14.725966 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8c8fd9c8-fae9-474c-a984-f0e815c379bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:14.786730 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1439/2851] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:14.796770 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:14.798234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:14.854980 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a12cc7-ee36-4e5e-8535-121d5b1b047f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:14.858557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb80d17-f057-4728-b9d2-e119536a00e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:14.859186 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb80d17-f057-4728-b9d2-e119536a00e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:14.859462 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb80d17-f057-4728-b9d2-e119536a00e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:14.859729 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb80d17-f057-4728-b9d2-e119536a00e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:14.868702 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb80d17-f057-4728-b9d2-e119536a00e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:14.870815 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb80d17-f057-4728-b9d2-e119536a00e2 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:14.880380 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1413/2852] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:18:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:18:15.270711 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fd9c8-fae9-474c-a984-f0e815c379bd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:15.272123 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8c8fd9c8-fae9-474c-a984-f0e815c379bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:15.286853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fd9c8-fae9-474c-a984-f0e815c379bd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:15.292288 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:15.293154 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:15.293440 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:15.293715 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] PATH_INFO: `/v3/projects/2bf14af935074201ac07949b467b3c99` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:15.295050 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf14af935074201ac07949b467b3c99)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:15.297944 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:15.418645 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf14af935074201ac07949b467b3c99', 'request_id': 'req-098378ac-d9ff-4636-a97f-d9a86c2a696b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:15.424076 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf14af935074201ac07949b467b3c99', 'request_id': 'req-098378ac-d9ff-4636-a97f-d9a86c2a696b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:15.430324 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf14af935074201ac07949b467b3c99', 'request_id': 'req-098378ac-d9ff-4636-a97f-d9a86c2a696b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:15.493593 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] The token cache is being invalidate because project 2bf14af935074201ac07949b467b3c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:18:15.493881 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:15.494650 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-098378ac-d9ff-4636-a97f-d9a86c2a696b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:18:15.496131 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1440/2853] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:18:15 2022] DELETE /identity/v3/projects/2bf14af935074201ac07949b467b3c99 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:18:15.501078 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfef2d-8eed-465f-8418-6d77c0e26d57 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:15.506064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfef2d-8eed-465f-8418-6d77c0e26d57 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:15.506064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfef2d-8eed-465f-8418-6d77c0e26d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:15.519921 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-84bfef2d-8eed-465f-8418-6d77c0e26d57 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:15.521104 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-84bfef2d-8eed-465f-8418-6d77c0e26d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:15.580650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1414/2854] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:15.590146 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb9be-b083-4538-bb46-f37161edc4c6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:15.590461 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb9be-b083-4538-bb46-f37161edc4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:15.590743 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb9be-b083-4538-bb46-f37161edc4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:15.609397 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ccadb9be-b083-4538-bb46-f37161edc4c6 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:15.610619 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ccadb9be-b083-4538-bb46-f37161edc4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:15.618393 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1441/2855] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:15.622592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfef2d-8eed-465f-8418-6d77c0e26d57 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:15.632705 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfef2d-8eed-465f-8418-6d77c0e26d57 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:15.638089 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b245ac5-f791-4f9d-bb04-34a363e07355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:15.638739 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b245ac5-f791-4f9d-bb04-34a363e07355 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:15.639024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b245ac5-f791-4f9d-bb04-34a363e07355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:15.639302 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b245ac5-f791-4f9d-bb04-34a363e07355 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:15.640029 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b245ac5-f791-4f9d-bb04-34a363e07355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:18:15.640336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b245ac5-f791-4f9d-bb04-34a363e07355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:15.641919 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b245ac5-f791-4f9d-bb04-34a363e07355 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:15.654172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1415/2856] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 09:18:15.658477 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadb9be-b083-4538-bb46-f37161edc4c6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:15.667846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadb9be-b083-4538-bb46-f37161edc4c6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:15.676465 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae5ced-f75e-4991-8801-678812415aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:15.677068 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae5ced-f75e-4991-8801-678812415aae admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:15.677351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae5ced-f75e-4991-8801-678812415aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:15.677628 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae5ced-f75e-4991-8801-678812415aae admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:15.678367 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae5ced-f75e-4991-8801-678812415aae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:15.680428 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae5ced-f75e-4991-8801-678812415aae admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:15.741558 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1442/2857] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:18:15 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:15.797671 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b245ac5-f791-4f9d-bb04-34a363e07355 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:15.808376 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b245ac5-f791-4f9d-bb04-34a363e07355 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:15.851992 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:15.863708 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251cb60-3f5e-43b2-aacd-bd9563999338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:15.864374 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251cb60-3f5e-43b2-aacd-bd9563999338 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:15.864658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251cb60-3f5e-43b2-aacd-bd9563999338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:15.864932 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251cb60-3f5e-43b2-aacd-bd9563999338 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:15.865658 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251cb60-3f5e-43b2-aacd-bd9563999338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:15.867592 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251cb60-3f5e-43b2-aacd-bd9563999338 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:15.940364 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1416/2858] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:18:15 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:18:15.944464 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae5ced-f75e-4991-8801-678812415aae admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:15.955495 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae5ced-f75e-4991-8801-678812415aae admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:15.960024 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b14fe4-627d-409c-ab94-91ed02705ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:15.960674 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14fe4-627d-409c-ab94-91ed02705ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:15.960959 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14fe4-627d-409c-ab94-91ed02705ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:15.961235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14fe4-627d-409c-ab94-91ed02705ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:15.961957 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b14fe4-627d-409c-ab94-91ed02705ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:15.963601 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b14fe4-627d-409c-ab94-91ed02705ed2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:15.973888 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1443/2859] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:15.977650 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251cb60-3f5e-43b2-aacd-bd9563999338 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:15.987843 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251cb60-3f5e-43b2-aacd-bd9563999338 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:15.992511 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b789ea3-967e-44b3-89e9-1286f33b31b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:15.993174 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b789ea3-967e-44b3-89e9-1286f33b31b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:15.993450 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b789ea3-967e-44b3-89e9-1286f33b31b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:15.993723 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b789ea3-967e-44b3-89e9-1286f33b31b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:15.994965 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b789ea3-967e-44b3-89e9-1286f33b31b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:15.996522 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b789ea3-967e-44b3-89e9-1286f33b31b6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:16.007227 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1417/2860] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:18:16.011932 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b14fe4-627d-409c-ab94-91ed02705ed2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:16.021905 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b14fe4-627d-409c-ab94-91ed02705ed2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:16.026424 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74700bf-255a-4760-9f06-067744d3a44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:16.027274 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74700bf-255a-4760-9f06-067744d3a44d admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:16.027555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74700bf-255a-4760-9f06-067744d3a44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:16.027830 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74700bf-255a-4760-9f06-067744d3a44d admin admin] PATH_INFO: `/v3/projects/8a83c88f502a46e0bf36ce97e86398ce/users/043c93f3631a458d98654fae9140807a/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:16.042150 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74700bf-255a-4760-9f06-067744d3a44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a83c88f502a46e0bf36ce97e86398ce, user_id=043c93f3631a458d98654fae9140807a, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:16.044778 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74700bf-255a-4760-9f06-067744d3a44d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:16.128028 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1444/2861] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:18:16 2022] PUT /identity/v3/projects/8a83c88f502a46e0bf36ce97e86398ce/users/043c93f3631a458d98654fae9140807a/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:18:16.133435 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4b2e6-ed2c-4e74-a721-b54bab55246e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:16.133819 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4b2e6-ed2c-4e74-a721-b54bab55246e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:16.134162 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4b2e6-ed2c-4e74-a721-b54bab55246e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:16.154480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5fb4b2e6-ed2c-4e74-a721-b54bab55246e None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:16.155661 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5fb4b2e6-ed2c-4e74-a721-b54bab55246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:16.212835 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1418/2862] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:16.222282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74700bf-255a-4760-9f06-067744d3a44d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:16.223955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e74700bf-255a-4760-9f06-067744d3a44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:16.238553 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:16.280235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74700bf-255a-4760-9f06-067744d3a44d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:16.283845 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bb29d-4bec-41fa-99d4-f070f6dd8c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:16.284478 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bb29d-4bec-41fa-99d4-f070f6dd8c4f service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:16.284757 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bb29d-4bec-41fa-99d4-f070f6dd8c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:16.285031 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bb29d-4bec-41fa-99d4-f070f6dd8c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:16.294544 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bb29d-4bec-41fa-99d4-f070f6dd8c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:16.295956 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bb29d-4bec-41fa-99d4-f070f6dd8c4f service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:16.305895 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1445/2863] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:18:17.520753 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128150a-6ec3-4566-9ead-17715a09034b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:17.521308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128150a-6ec3-4566-9ead-17715a09034b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:17.521308 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128150a-6ec3-4566-9ead-17715a09034b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:17.550352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3128150a-6ec3-4566-9ead-17715a09034b None None] MFA Rules not processed for user `043c93f3631a458d98654fae9140807a`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:17.551606 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3128150a-6ec3-4566-9ead-17715a09034b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:17.617611 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1419/2864] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:17.630228 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bb29d-4bec-41fa-99d4-f070f6dd8c4f service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:17.631631 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fd3bb29d-4bec-41fa-99d4-f070f6dd8c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:17.689069 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bb29d-4bec-41fa-99d4-f070f6dd8c4f service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:17.692646 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45094d39-63bb-48ac-80dd-08f4282aa1ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:17.693284 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45094d39-63bb-48ac-80dd-08f4282aa1ee service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:17.693568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45094d39-63bb-48ac-80dd-08f4282aa1ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:17.693854 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45094d39-63bb-48ac-80dd-08f4282aa1ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:17.702139 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45094d39-63bb-48ac-80dd-08f4282aa1ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:17.703560 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45094d39-63bb-48ac-80dd-08f4282aa1ee service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:17.712349 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1446/2865] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:18:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:18:19.447099 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb6e4f-beba-4cb8-8b70-f63285c18f94 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:19.447614 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb6e4f-beba-4cb8-8b70-f63285c18f94 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:19.447717 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb6e4f-beba-4cb8-8b70-f63285c18f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:19.465043 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-25eb6e4f-beba-4cb8-8b70-f63285c18f94 None None] MFA Rules not processed for user `043c93f3631a458d98654fae9140807a`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:19.466790 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-25eb6e4f-beba-4cb8-8b70-f63285c18f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:19.473712 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1420/2866] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:27.708344 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=46 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:29.459693 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:36.532673 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e38957-a174-40b0-b7e6-58f2d49cdf16 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:36.533169 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e38957-a174-40b0-b7e6-58f2d49cdf16 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:36.533275 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e38957-a174-40b0-b7e6-58f2d49cdf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:36.550546 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-74e38957-a174-40b0-b7e6-58f2d49cdf16 None None] MFA Rules not processed for user `043c93f3631a458d98654fae9140807a`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:36.551737 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-74e38957-a174-40b0-b7e6-58f2d49cdf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:36.558669 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1447/2867] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:18:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:46.545242 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:18:59.373024 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fbe4b-6795-4b03-bd12-9319b9337212 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:59.373515 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fbe4b-6795-4b03-bd12-9319b9337212 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:59.373623 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fbe4b-6795-4b03-bd12-9319b9337212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:59.422084 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-990fbe4b-6795-4b03-bd12-9319b9337212 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:18:59.423455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-990fbe4b-6795-4b03-bd12-9319b9337212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:59.489258 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1421/2868] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:18:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:18:59.629747 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e38957-a174-40b0-b7e6-58f2d49cdf16 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:18:59.635788 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-74e38957-a174-40b0-b7e6-58f2d49cdf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:18:59.721491 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e38957-a174-40b0-b7e6-58f2d49cdf16 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:18:59.726621 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4346d-678d-4bb3-8455-b5a10e8ebb4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:18:59.727262 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4346d-678d-4bb3-8455-b5a10e8ebb4e service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:18:59.727542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4346d-678d-4bb3-8455-b5a10e8ebb4e service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:18:59.727818 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4346d-678d-4bb3-8455-b5a10e8ebb4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:18:59.735832 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4346d-678d-4bb3-8455-b5a10e8ebb4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:18:59.737411 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4346d-678d-4bb3-8455-b5a10e8ebb4e service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:18:59.746153 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1448/2869] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:18:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:19:09.485976 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=40 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:19:09.742659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=41 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:19:16.226350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489a66d-1a32-4c9e-84b7-c051c663ff06 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:16.226787 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489a66d-1a32-4c9e-84b7-c051c663ff06 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:16.226900 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489a66d-1a32-4c9e-84b7-c051c663ff06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:16.245033 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9489a66d-1a32-4c9e-84b7-c051c663ff06 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:19:16.246263 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9489a66d-1a32-4c9e-84b7-c051c663ff06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:16.253557 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1422/2870] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 13 09:19:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:19:16.265676 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4346d-678d-4bb3-8455-b5a10e8ebb4e service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:16.277048 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4346d-678d-4bb3-8455-b5a10e8ebb4e service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:16.282105 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f272323-deb5-4d15-bedd-7c2e6c1bccb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:16.282770 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272323-deb5-4d15-bedd-7c2e6c1bccb2 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:16.283055 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272323-deb5-4d15-bedd-7c2e6c1bccb2 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:16.283331 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272323-deb5-4d15-bedd-7c2e6c1bccb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:16.292152 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f272323-deb5-4d15-bedd-7c2e6c1bccb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:16.293736 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f272323-deb5-4d15-bedd-7c2e6c1bccb2 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:16.301767 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1449/2871] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:19:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:19:18.972487 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489a66d-1a32-4c9e-84b7-c051c663ff06 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:18.974010 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9489a66d-1a32-4c9e-84b7-c051c663ff06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:19.044234 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489a66d-1a32-4c9e-84b7-c051c663ff06 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:19.049241 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c6b6f-d0ff-4ca9-bc0e-3a825e3427bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:19.049875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c6b6f-d0ff-4ca9-bc0e-3a825e3427bd service placement] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:19.050189 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c6b6f-d0ff-4ca9-bc0e-3a825e3427bd service placement] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:19.050458 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c6b6f-d0ff-4ca9-bc0e-3a825e3427bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:19.052298 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c65c6b6f-d0ff-4ca9-bc0e-3a825e3427bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:19.063518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c6b6f-d0ff-4ca9-bc0e-3a825e3427bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:19.065064 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c6b6f-d0ff-4ca9-bc0e-3a825e3427bd service placement] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:19.075593 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1423/2872] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:19:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:19:19.090938 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f272323-deb5-4d15-bedd-7c2e6c1bccb2 service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:19.101898 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f272323-deb5-4d15-bedd-7c2e6c1bccb2 service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:19.106781 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:19.107892 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:19.108180 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:19.108453 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] PATH_INFO: `/v3/users/043c93f3631a458d98654fae9140807a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:19.109698 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=043c93f3631a458d98654fae9140807a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:19.111719 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:19.284121 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043c93f3631a458d98654fae9140807a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:19.291001 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043c93f3631a458d98654fae9140807a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:19.350623 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043c93f3631a458d98654fae9140807a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:19.351235 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043c93f3631a458d98654fae9140807a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:19.351853 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6df93691-d7b0-4808-829f-de5a03f66381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043c93f3631a458d98654fae9140807a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:19.370259 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1450/2873] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:19:19 2022] DELETE /identity/v3/users/043c93f3631a458d98654fae9140807a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:19:19.588799 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c6b6f-d0ff-4ca9-bc0e-3a825e3427bd service placement] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:19.601201 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c6b6f-d0ff-4ca9-bc0e-3a825e3427bd service placement] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:19.660050 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:19.660984 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:19.661261 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:19.662066 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] PATH_INFO: `/v3/projects/8a83c88f502a46e0bf36ce97e86398ce` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:19.663246 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a83c88f502a46e0bf36ce97e86398ce)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:19.665155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:19.784677 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a83c88f502a46e0bf36ce97e86398ce', 'request_id': 'req-a60e34cc-6436-46a3-ad07-6ad16168f55b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:19.791088 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a83c88f502a46e0bf36ce97e86398ce', 'request_id': 'req-a60e34cc-6436-46a3-ad07-6ad16168f55b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:19.876327 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a83c88f502a46e0bf36ce97e86398ce', 'request_id': 'req-a60e34cc-6436-46a3-ad07-6ad16168f55b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:19.887621 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] The token cache is being invalidate because project 8a83c88f502a46e0bf36ce97e86398ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:19:19.893822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:19.893822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a60e34cc-6436-46a3-ad07-6ad16168f55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:19.893822 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1424/2874] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:19:19 2022] DELETE /identity/v3/projects/8a83c88f502a46e0bf36ce97e86398ce => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:19:19.896184 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05e348-291f-475f-a2ba-2d5ad9485662 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:19.896779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05e348-291f-475f-a2ba-2d5ad9485662 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:19.896779 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05e348-291f-475f-a2ba-2d5ad9485662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:19.916222 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ab05e348-291f-475f-a2ba-2d5ad9485662 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:19:19.917408 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ab05e348-291f-475f-a2ba-2d5ad9485662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:19.983364 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1451/2875] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:19:19.991001 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7998de-8dba-4343-97f7-066d3ef0399b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:19.991480 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7998de-8dba-4343-97f7-066d3ef0399b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:19.991578 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7998de-8dba-4343-97f7-066d3ef0399b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:20.009944 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-aa7998de-8dba-4343-97f7-066d3ef0399b None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:19:20.011172 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-aa7998de-8dba-4343-97f7-066d3ef0399b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:20.019042 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1425/2876] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:19:20.023583 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05e348-291f-475f-a2ba-2d5ad9485662 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:20.033065 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05e348-291f-475f-a2ba-2d5ad9485662 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:20.038286 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df1076-56e8-4123-8fe0-92ef4f7117d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:20.038923 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1076-56e8-4123-8fe0-92ef4f7117d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:20.039201 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1076-56e8-4123-8fe0-92ef4f7117d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:20.039472 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1076-56e8-4123-8fe0-92ef4f7117d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:20.040186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df1076-56e8-4123-8fe0-92ef4f7117d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 09:19:20.040486 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df1076-56e8-4123-8fe0-92ef4f7117d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:20.042089 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df1076-56e8-4123-8fe0-92ef4f7117d7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:20.053605 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1452/2877] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 13 09:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 09:19:20.057638 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7998de-8dba-4343-97f7-066d3ef0399b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:20.066713 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7998de-8dba-4343-97f7-066d3ef0399b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:20.071518 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a19b8-adc1-4465-9176-5df752673f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:20.072156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a19b8-adc1-4465-9176-5df752673f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:20.072439 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a19b8-adc1-4465-9176-5df752673f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:20.072715 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a19b8-adc1-4465-9176-5df752673f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:20.073428 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a19b8-adc1-4465-9176-5df752673f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:20.075336 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a19b8-adc1-4465-9176-5df752673f7f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:20.078571 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:19:20.186786 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1426/2878] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 13 09:19:20 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:19:20.190566 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df1076-56e8-4123-8fe0-92ef4f7117d7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:20.201003 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df1076-56e8-4123-8fe0-92ef4f7117d7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:20.245691 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:19:20.258618 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aef0a-d785-4eec-8698-802d194d0a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:20.259283 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aef0a-d785-4eec-8698-802d194d0a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:20.259568 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aef0a-d785-4eec-8698-802d194d0a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:20.259848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aef0a-d785-4eec-8698-802d194d0a13 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:20.260569 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4aef0a-d785-4eec-8698-802d194d0a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:20.262659 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4aef0a-d785-4eec-8698-802d194d0a13 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:20.354011 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1453/2879] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 13 09:19:20 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 09:19:20.358181 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a19b8-adc1-4465-9176-5df752673f7f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:20.368607 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a19b8-adc1-4465-9176-5df752673f7f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:20.373346 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970f98c-4a86-453d-97ed-54a8f3b682f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:20.374055 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f98c-4a86-453d-97ed-54a8f3b682f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:20.374352 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f98c-4a86-453d-97ed-54a8f3b682f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:20.374670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f98c-4a86-453d-97ed-54a8f3b682f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:20.375395 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970f98c-4a86-453d-97ed-54a8f3b682f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:20.377253 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970f98c-4a86-453d-97ed-54a8f3b682f2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:20.387683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1427/2880] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:19:20.391681 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aef0a-d785-4eec-8698-802d194d0a13 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:20.402265 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aef0a-d785-4eec-8698-802d194d0a13 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:20.406986 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf20741-12b4-4c2b-b981-7fe7a50208be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:20.407634 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf20741-12b4-4c2b-b981-7fe7a50208be admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:20.407919 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf20741-12b4-4c2b-b981-7fe7a50208be admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:20.408195 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf20741-12b4-4c2b-b981-7fe7a50208be admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:20.408921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf20741-12b4-4c2b-b981-7fe7a50208be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:20.410921 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf20741-12b4-4c2b-b981-7fe7a50208be admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:20.421823 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1454/2881] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 13 09:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 09:19:20.425886 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970f98c-4a86-453d-97ed-54a8f3b682f2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:20.436180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970f98c-4a86-453d-97ed-54a8f3b682f2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:20.440857 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a132fd-6991-42a0-99c8-ef3acde6a8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:20.441670 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a132fd-6991-42a0-99c8-ef3acde6a8a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:20.441949 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a132fd-6991-42a0-99c8-ef3acde6a8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:20.442257 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a132fd-6991-42a0-99c8-ef3acde6a8a6 admin admin] PATH_INFO: `/v3/projects/cf90881aef9b4222bdd3ca3c3ea67fd9/users/78a178a38d1945c59876dad28ec73a66/roles/a70b63ede7014fa9bbcb4e03275a2664` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:20.457546 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a132fd-6991-42a0-99c8-ef3acde6a8a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf90881aef9b4222bdd3ca3c3ea67fd9, user_id=78a178a38d1945c59876dad28ec73a66, role_id=a70b63ede7014fa9bbcb4e03275a2664)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:20.460829 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a132fd-6991-42a0-99c8-ef3acde6a8a6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:20.519404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1428/2882] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 13 09:19:20 2022] PUT /identity/v3/projects/cf90881aef9b4222bdd3ca3c3ea67fd9/users/78a178a38d1945c59876dad28ec73a66/roles/a70b63ede7014fa9bbcb4e03275a2664 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 09:19:20.531035 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c812d53-303c-4085-b347-db1f9279d202 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:20.531330 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c812d53-303c-4085-b347-db1f9279d202 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:20.531612 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c812d53-303c-4085-b347-db1f9279d202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:20.557597 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6c812d53-303c-4085-b347-db1f9279d202 None None] MFA Rules not processed for user `78a178a38d1945c59876dad28ec73a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:19:20.558844 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6c812d53-303c-4085-b347-db1f9279d202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:20.623962 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1455/2883] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:19:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:19:20.635350 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a132fd-6991-42a0-99c8-ef3acde6a8a6 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:20.639629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-01a132fd-6991-42a0-99c8-ef3acde6a8a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:20.694068 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a132fd-6991-42a0-99c8-ef3acde6a8a6 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:20.697826 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d325084-82f0-445e-8cb9-7b214c4b2a31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3f12750f91645748b37e2b159a9bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:20.698496 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d325084-82f0-445e-8cb9-7b214c4b2a31 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:20.698850 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d325084-82f0-445e-8cb9-7b214c4b2a31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:20.699123 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d325084-82f0-445e-8cb9-7b214c4b2a31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:20.707629 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d325084-82f0-445e-8cb9-7b214c4b2a31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:20.709243 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d325084-82f0-445e-8cb9-7b214c4b2a31 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:20.718002 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1429/2884] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:19:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:19:22.384731 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d75a4-8ec6-4745-b393-1a6560ba14ae None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:22.385232 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d75a4-8ec6-4745-b393-1a6560ba14ae None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:22.385351 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d75a4-8ec6-4745-b393-1a6560ba14ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:22.402507 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e46d75a4-8ec6-4745-b393-1a6560ba14ae None None] MFA Rules not processed for user `78a178a38d1945c59876dad28ec73a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:19:22.403689 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e46d75a4-8ec6-4745-b393-1a6560ba14ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:22.410449 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1456/2885] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:19:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:19:24.267555 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d325084-82f0-445e-8cb9-7b214c4b2a31 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:24.268938 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8d325084-82f0-445e-8cb9-7b214c4b2a31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:24.338426 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d325084-82f0-445e-8cb9-7b214c4b2a31 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:24.344052 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046378f-495a-4f98-9277-e396432fba85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bca7c1164d3e4833a10ffc37cbc6f092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:24.344704 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046378f-495a-4f98-9277-e396432fba85 service placement] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:24.344986 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046378f-495a-4f98-9277-e396432fba85 service placement] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:24.345284 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046378f-495a-4f98-9277-e396432fba85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:24.347098 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6046378f-495a-4f98-9277-e396432fba85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:24.359616 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046378f-495a-4f98-9277-e396432fba85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:24.361180 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046378f-495a-4f98-9277-e396432fba85 service placement] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:24.371816 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1430/2886] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:19:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:19:32.397293 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:19:34.367540 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=79,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:19:39.760786 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de8196-42db-405b-a346-609a555c13be None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:39.761282 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de8196-42db-405b-a346-609a555c13be None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:39.761391 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de8196-42db-405b-a346-609a555c13be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:39.779458 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e4de8196-42db-405b-a346-609a555c13be None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:19:39.780643 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e4de8196-42db-405b-a346-609a555c13be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:39.832982 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1457/2887] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:19:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:19:39.847868 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046378f-495a-4f98-9277-e396432fba85 service placement] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:39.849294 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6046378f-495a-4f98-9277-e396432fba85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:39.917701 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046378f-495a-4f98-9277-e396432fba85 service placement] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:39.922572 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281a157-98d2-4a9e-b04d-bfb70b728c53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:39.923205 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281a157-98d2-4a9e-b04d-bfb70b728c53 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:39.923483 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281a157-98d2-4a9e-b04d-bfb70b728c53 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:39.923755 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281a157-98d2-4a9e-b04d-bfb70b728c53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:39.932282 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2281a157-98d2-4a9e-b04d-bfb70b728c53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:39.933846 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2281a157-98d2-4a9e-b04d-bfb70b728c53 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:39.941695 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1431/2888] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:19:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:19:40.858085 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501208d1-e28c-40ee-8d25-c6acb5ac9650 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:40.858564 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501208d1-e28c-40ee-8d25-c6acb5ac9650 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:40.858679 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501208d1-e28c-40ee-8d25-c6acb5ac9650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:40.877179 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-501208d1-e28c-40ee-8d25-c6acb5ac9650 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:19:40.878402 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-501208d1-e28c-40ee-8d25-c6acb5ac9650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:40.885579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1458/2889] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 13 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:19:40.897502 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281a157-98d2-4a9e-b04d-bfb70b728c53 service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:40.908596 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281a157-98d2-4a9e-b04d-bfb70b728c53 service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:40.914095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309e74-8039-4a25-9923-6668a9aeb1bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:40.914750 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309e74-8039-4a25-9923-6668a9aeb1bd service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:40.915025 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309e74-8039-4a25-9923-6668a9aeb1bd service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:40.915293 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309e74-8039-4a25-9923-6668a9aeb1bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:40.923131 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309e74-8039-4a25-9923-6668a9aeb1bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:40.924683 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309e74-8039-4a25-9923-6668a9aeb1bd service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:40.932805 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1432/2890] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:19:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:19:41.580653 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc7cdb-9020-4acf-9a2c-950404b94d3a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:41.580940 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc7cdb-9020-4acf-9a2c-950404b94d3a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:41.581219 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc7cdb-9020-4acf-9a2c-950404b94d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:41.601863 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-15cc7cdb-9020-4acf-9a2c-950404b94d3a None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:19:41.603026 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-15cc7cdb-9020-4acf-9a2c-950404b94d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:41.611209 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1459/2891] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 13 09:19:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:19:41.625754 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309e74-8039-4a25-9923-6668a9aeb1bd service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:41.637354 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309e74-8039-4a25-9923-6668a9aeb1bd service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:41.642606 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d49480-ab01-464f-ad39-1340ff8e08fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:41.643248 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d49480-ab01-464f-ad39-1340ff8e08fd service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:41.643523 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d49480-ab01-464f-ad39-1340ff8e08fd service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:41.643794 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d49480-ab01-464f-ad39-1340ff8e08fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:41.653109 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d49480-ab01-464f-ad39-1340ff8e08fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:41.655780 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d49480-ab01-464f-ad39-1340ff8e08fd service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:41.664811 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1433/2892] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:19:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 09:19:42.245613 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9393bb6-0dec-47a5-972b-be760f14e7c9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:42.246134 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9393bb6-0dec-47a5-972b-be760f14e7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:42.246365 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9393bb6-0dec-47a5-972b-be760f14e7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:42.264635 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f9393bb6-0dec-47a5-972b-be760f14e7c9 None None] MFA Rules not processed for user `0b380bc85da64a8bb687e2e5c4577e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:19:42.265846 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f9393bb6-0dec-47a5-972b-be760f14e7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:42.273549 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1460/2893] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 13 09:19:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:19:42.285512 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d49480-ab01-464f-ad39-1340ff8e08fd service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:42.296766 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d49480-ab01-464f-ad39-1340ff8e08fd service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:42.302145 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db929b12-7f22-4b36-9131-e6a8eba8dc4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '906fc971a5ee4ffd9503ab13d46cd605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:42.302779 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db929b12-7f22-4b36-9131-e6a8eba8dc4f service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:42.303095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db929b12-7f22-4b36-9131-e6a8eba8dc4f service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:42.303370 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db929b12-7f22-4b36-9131-e6a8eba8dc4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:42.312188 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db929b12-7f22-4b36-9131-e6a8eba8dc4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:42.313797 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db929b12-7f22-4b36-9131-e6a8eba8dc4f service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:42.322095 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1434/2894] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 13 09:19:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 09:19:44.855500 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9393bb6-0dec-47a5-972b-be760f14e7c9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:44.917542 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9393bb6-0dec-47a5-972b-be760f14e7c9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:44.921955 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:44.922848 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:44.923125 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:44.923395 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] PATH_INFO: `/v3/users/78a178a38d1945c59876dad28ec73a66` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:44.924579 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a178a38d1945c59876dad28ec73a66)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:44.926555 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:45.073417 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a178a38d1945c59876dad28ec73a66'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:45.080173 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a178a38d1945c59876dad28ec73a66'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:45.150238 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a178a38d1945c59876dad28ec73a66'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:45.150860 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a178a38d1945c59876dad28ec73a66'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:45.151457 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a178a38d1945c59876dad28ec73a66'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:45.169754 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1461/2895] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 13 09:19:44 2022] DELETE /identity/v3/users/78a178a38d1945c59876dad28ec73a66 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:19:45.174210 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a964f0d-f588-4013-a468-abdc41d816c8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:45.174688 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a964f0d-f588-4013-a468-abdc41d816c8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:45.174915 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a964f0d-f588-4013-a468-abdc41d816c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:45.192156 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0a964f0d-f588-4013-a468-abdc41d816c8 None None] MFA Rules not processed for user `26ad26ac4aea4a7b8fb923b47a154aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 09:19:45.193368 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0a964f0d-f588-4013-a468-abdc41d816c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:45.250875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1435/2896] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 13 09:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 09:19:45.261186 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:45.262656 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:45.269272 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:19:45.319099 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36901c0e-8478-40d1-9289-4942538b24c9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:45.323481 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d2bf9-454c-4548-9be6-ba21c7f0e33a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12824152139841138f457dfe5c9758e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa56bd5dbb81408c8fe362c5a1c02105', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:45.324124 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d2bf9-454c-4548-9be6-ba21c7f0e33a service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:45.324409 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d2bf9-454c-4548-9be6-ba21c7f0e33a service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:45.324687 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d2bf9-454c-4548-9be6-ba21c7f0e33a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:45.334576 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d2bf9-454c-4548-9be6-ba21c7f0e33a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:45.336761 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d2bf9-454c-4548-9be6-ba21c7f0e33a service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:45.347031 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1462/2897] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 13 09:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 09:19:45.574903 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a964f0d-f588-4013-a468-abdc41d816c8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 09:19:45.582247 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0a964f0d-f588-4013-a468-abdc41d816c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 09:19:45.588404 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a964f0d-f588-4013-a468-abdc41d816c8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 09:19:45.593132 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ad26ac4aea4a7b8fb923b47a154aa9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8052949c90742f6ada3f8a34d5636f8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 09:19:45.594224 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 09:19:45.594700 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 09:19:45.594979 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] PATH_INFO: `/v3/projects/cf90881aef9b4222bdd3ca3c3ea67fd9` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 09:19:45.596171 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf90881aef9b4222bdd3ca3c3ea67fd9)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 09:19:45.598116 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 09:19:45.737385 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:19:45.740107 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf90881aef9b4222bdd3ca3c3ea67fd9', 'request_id': 'req-e29012b1-c34a-4ce3-ae86-4faaa3746cab'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:45.746344 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf90881aef9b4222bdd3ca3c3ea67fd9', 'request_id': 'req-e29012b1-c34a-4ce3-ae86-4faaa3746cab'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:45.861311 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf90881aef9b4222bdd3ca3c3ea67fd9', 'request_id': 'req-e29012b1-c34a-4ce3-ae86-4faaa3746cab'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:45.874685 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] The token cache is being invalidate because project cf90881aef9b4222bdd3ca3c3ea67fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 09:19:45.875225 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:45.875875 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e29012b1-c34a-4ce3-ae86-4faaa3746cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 09:19:45.877455 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1436/2898] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 13 09:19:45 2022] DELETE /identity/v3/projects/cf90881aef9b4222bdd3ca3c3ea67fd9 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 09:19:55.361821 dv-cinder-849626-3-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=24 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 09:19:55.890155 dv-cinder-849626-3-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}